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ABSTRACT 

The increasing number of medical centers involved in 
collaborative and innovative health services in the community is but 
one reflection of social concerns and pressures for change in the 
health care system. Medical schools and their affiliated teaching 
hospitals are trying in various ways to adapt their teaching, 
research, and service functions to these concerns. With the intent of 
exploring how a number of medical centers are approaching these 
problems, the Harvard Center for Community Health and Medical Care in 
the spring of 1970 sponsored an Invitational Conference on the 
Evaluation of University Medical Care Programs. The conference 
brought together representatives of 6 medical schools with 
comprehensive medical care plans, as well as people with special 
backgrounds in medical care, economics, prepaid heal'th care plans, 
and program evaluation. The specific thrust of the conference 
centered around how each of the participating medical schools are 
proposing to evaluate their particular medical care plan. This 
document presents the papers and discussions that took place at the 
conference. (Author/HS) 
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FOREWORD 



The increasing number of medical centers involved in collaborative and 
innovative health services in the community is but one reflection of social 
concerns and pressures for change in the health care system. Medical schools 
and their affiliated teaching hospitals are trying in various ways to adapt their 
teaching, research and service functions to these concerns. 

With the intent of exploring how a number of medical centers are 
approaching these problems, the Harvard Center for Community Health and 
Medical Care in the Spring of 1970 sponsored an Invitational Conference on 
the Evaluation of University Medical Care Programs.* The Conference brought 
together representatives of six medical schools with comprehensive medical 
care plans, as well as people with special backgrounds in medical care, 
economics, prepaid health care plans, and program evaluation. 

While many conferences have been held on the general theory and art of 
evaluation, this conference sought a sharper focus: specifically how were each 
of the participating medical schools proposing to evaluate their particular 
medical care plan? 

Throughout the conference the participants returned again and again to 
the need for clarification of each medical school’s objectives in extending its 
program into the community. Most medical schools and their affiliated 
hospitals provide care to large numbers of indigent people through their 
out-patient departments— particularly the emergency service. The extension of 
the out-patient department into the community for the purpose of reaching 
larger numbers of people is a laudable, humanitarian endeavor, but if it involves 
no essential change in the pattern of delivery of care, it perpetuates a system 
which is highly unsatisfactory to recipient and provider alike. Moreover, no 
medical school is in a position to assume responsibility for providing medical 
care to the population of entire communities such as Boston, Baltimore, New 
Haven, St. Louis, or other large metropolitan areas. Indeed one of the 
dilemmas of our time is that it is not clear what agency in the community in 
fact does have the responsibility and authority for insuring that the entire 
population has equal access to high quality care. What then is a meaningful role 
for the medical school in the community? 

♦This conference was supported by Grant HS 00632-01 from the National Center 
for Health Services Research and Development, HSMHA, DHEW. 
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The medical school can, perhaps, best serve the community by opening 
up new options through the development and testing of different ways of 
organizing and paying for the delivery of medical care. Indeed such a role is in 
keeping with its primary function as an educational institution. 

However, the development of experiments in the delivery and financing 
of health services is linked with the responsibility for assessing the significance 
of the experiments both in contributing to the care of the individual patient 
and in meeting the needs of the community. The first requires, beyond 
traditional clinical research, new understanding of how the process of care 
itself may affect the individual patient’s prognosis-such as the influence of 
different manpower mixes on the incidence of complications or other 
outcomes. The second requires careful definition of the population to be 
served and a thorough assessment of the extent to which this population’s 
needs are met. Systematic analysis of this second area requires information 
about those segments of the population which are not served as well as those 
which are. This means that the unit of evaluative study must be the community 
as well as the patient. If this is not so, the opportunity for the medical school 
to delineate the options available to the community in its efforts to meet its 
needs will be lost. 

In its educational role the medical school also views its plan for providing 
health services in the community as a setting for the education of medical 
students and house staff in the problems and challenges of delivering health 
care outside the hospital setting. Most medical care, after all, is provided on an 
ambulatory basis. For the educational process to have meaning beyond the 
exigencies of the moment, both the rationale which underlies the care of the 
individual patient and that which dictates the structure of the delivery of 
services must be under constant review. Again we are led to the need for 
systematic study of the health care process both from the individual and 
community standpoint. It is perhaps here that medical schools can make a 
unique contribution since they have the potential for linking the educational 
and service functions in an integral fashion through effective evaluation. 

The medical schools participating in the conference reported here are 
acutely aware of these issues. It is in the belief that their struggles to grapple 
with such issues will be of value to others facing similar problems that these 
Proceedings are presented. 9 



Paul M. Densen, Director 
Harvard Center for Community Health 
and Medical Care 
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PRELIMINARY SESSION 



In the early phase of planning the Conference, medical center representa- 
tives requested a preliminary session to review and discuss pressing issues in 
organizing, administering, and financing the plans. The plea for this additional 
time was generated from background materials prepared by each center and 
distributed prior to the Conference. This background information served as the 
focus of discussion during the preliminary session at which time fundamentals 
of the plans were described and attention was directed to the multiple, and 
often diverse, objectives among the plans. Substantial differences in the 
organizational structures and financing arrangements* were noted, and some of 
the expected changes in delivery of medical care and in the focus of medical 
education were underlined. 

Impetus for Developing the Plans 

In spite of the diverse objectives mentioned above, there was a common 
theme that seemed to motivate the formation cf these university medical care 
plans. This theme was a recognition of the need for adaptations in the health 
care system, and the desire of the universities to be in the vanguard of efforts 
to introduce and test new methods of providing medical care and to adapt 
medical education to these innovations. 

The medical schools and their affiliated hospitals, aware oi the widening 
gap between expectations and demands of the populations served and the 
ability of the health care systems to deliver the services needed, hope through 
the development of imaginative programs to demonstrate the changes required 
to provide a more effective delivery system and to suggest appropriate funding 
arrangements. In the belief that a monolithic health structure is inappropriate 
for the United States, many kinds of organizational forms and financial 
arrangements need to be developed. Under these circumstances small-scale 
experiments with different organizational methods and different financing 
arrangements will be particularly important. The university medical center 
provides an appropriate setting for testing and evaluating the relative 
effectiveness of these experiments. 

Objectives of the Plans 

Most plans stated several concurrent objectives, including three or more 
of the following: rendering good primary care to patients while making 
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specialist care available as needed; improving the health of the target 
population; providing a setting for teaching and training programs; experiment- 
ing with alternative manpower configurations for delivery of health services; 
determining the effect of alternative methods of financing on medical care 
processes and or. patients; developing a model of health delivery and/or 
financing; and providing a laboratory for health services research. 

Such multiple and broadly stated objectives present difficulties both in 
the development of specific medical care plans and in the assessment of the 
extent to which objectives are accomplished. The possibility of developing 
common measures of success in achieving objectives within a given center and 
between centers requires that objectives be stated as explicitly as possible and 
some degree of commonality of objectives be present among the centers. 

Discussion revealed that the diversities in purpose among some of the 
plans were due to exigencies such as state laws, university policy, government 
agency guidelines or other requirements of their prospective funding and 
enrollment sources, and the complex interplay of demands by both profes- 
sional and consumer organizations. The stated objectives of many groups were 
conditioned to some extent by one of more of these demands. However, some 
of these exigencies provided an impetus for change, and the medical centers 
began negotiations among sponsors, providers, and consumer groups which 
gave rise to the plans presented at the Conference. 

According different priorities to objectives influenced, to a great degree, 
each plan’s specific operational goals, and the selection of appropriate measures 
for evaluating its success. The discussions made clear that comparisons of the 
service or cost experience of various plans could bo made only after careful 
review of similarities and differences in major objectives. Ideally it would be 
desirable to evaluate the service, research and teaching functions separately. 
The methodological problems in this approach are enormous, but some of the 
plans are attempting to address this issue. 

Organizational Structure and Financing of the Plans 

The legal authority of each of the plans is vested in one of the following 
corporate structures: directly under university, medical school and/or teaching 
hospital auspices; a separate corporation spun off from the medical school and 
teaching hospitals, with formal associations maintained; an independent 
community corporation, contracting with a medical school for purchase of 
service and for participation in education, training, research and evaluation. 

Similarly, the de facto policy-making bodies varied in type, ranging from 
a committee composed of chairmen of the clinical departments of a medical 
school to boards composed of various mixes of medical school-hospital 
representatives, consumers, representatives of the general public, and insurance 
company sponsors. 
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In some plans, where boards have a majority of lay members, the 
selection and supervision of health professionals have been delegated to a 
special committee or group at the medical school, with veto power on 
individual appointments retained by the lay board. In others, where the plan is 
a chartered community corporation having formal affiliations with a univer- 
sity-medical center (e.g. the Community Plan in New Haven), the selection and 
appointment of health professionals are made initially by the corporation with 
consultation from the medical center to insure professionally qualified persons. 

Plans are financed through various combinations of federal funds (Title 
XVIII and XIX, OEO, giants, and others), group health insurance oiganizations 
(i.e. Blue Cross-Blue Shield and commercial insurance carriers), private 
companies, state and local agencies, and other schemes. Each organization, 
whether public or private, has its own set of regulations governing eligibility, 
enrollment, and payment for service. All of these factors directly influence the 
definition of the population exposed to risk, the services offered, and may 
affect administrative costs of the plans. 

Implications of Funding 

The characteristics of the population entering the plan determine the 
demands for service, influence staffing patterns, and order the kinds of services 
acceptable to these consumers. Thus it is important to note that the population 
to be served is determined to a great extent by the financing arrangements 
adopted. These funding arrangements also affect the nature of the record 
keeping system and therefore the nature of the evaluation. For example, if a 
plan obtains members through the Blue Cross group enrollment mechanism, an 
employment base is usual; if on the other hand OEO provides payment, 
eligibility is based on income guidelines; and if categorical grants are awarded, a 
specific type of health program is required. Where a plan receives capitation 
payments from multiple sources, it must contend with possibilities for 
confusion not only in the mechanism through which persons enter the plan but 
also in the methods by which the funds are allocated. Funding for one of the 
plans was provided after a series of negotiations with fourteen agencies, each 
with different requirements and modes of reimbursement; one agency would 
allow funds only for service performed by a recognized provider (an established 
institution), while another agency would grant funds only to a community 
group organized to insure community participation and financial responsibility 
for the program. Many of the plans have faced the difficult process of trying to 
work within these restrictions and have set up methods for receiving multiple 
funding and the accounting system necessary for the varied reporting 
requirements. 

The corporate structure of many of the universities did not permit, nor 
did many wish to become, underwriters of health insurance. Therefore, some 
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of the plans have negotiated contracts to provide stipulated health service 
benefits with the existing insurance carriers, the Blues and commercial 
companies, serving as the insuring agents. In most cases this meant that the 
existing carrier would offer a dual choice in those geographic areas where the 
university plans were available. This r *"‘.od of obtaining subscribers for the 
university medical care plans has pi >nted problems in areas where the 
employees of large companies are covered by national contracts for group 
insurance benefits. There also have been problems in obtaining dual choice 
options for civil servants of local, state, and federal governments under the 
present prepayment system.* 

In the discussions the question was raised concerning the relatively small 
rate of growth of prepaid group practice plans in the United States. This small 
growth rate leads one to consider issues involved in the enrollment process. 
Basically, in the past, there have been only two ways to enroll in prepayment 
group practice plans: 1) either through membership in large unions, or 2) 
employment in large industries. Yet, to be acceptable, simpler and more 
flexible enrollment methods are essential. The medical centers have been 
experimenting with different ways to obtain enrollees. Indeed one of the most 
innovative features of the university plans may be methods developed for 
marketing the prepaid group practice plan through existing insurance 
mechanisms. 

If medical care is to move in new directions and use new modes of 
financing, consideration must be given to innovative educational methods in 
the training of physicians and other health personnel wh*’ will be responsible 
for providing comprehensive health care in the medical center programs. In the 
past the prepayment group practice plans have jot had a part in the medical 
training of providers of service but have been dependent upon physicians, 
trained in the atmosphere of teaching hospitals and specialty medicine, who 
joined the plans out of social commitment. It is hoped that physicians trained 
in a different environment will view the delivery of health care within the 
conceptual framework of a total health system. 

The medical center plans, to a great extent, have faced similar 
difficulties. Many of their programs have evolved from medical schools and 
their affiliated hospitals working toward changing the modus operandi of 
health care from crisis intervention to a health care system in which die 
organizational dimensions become as important as the clinical. To be 
accomplished these changes require a major adjustment on the part of many 
physicians, who are essential to the medical schools and to the hospitals and 
whose judgment is crucial in the development of new models in the delivery of 
medical services. To succeed, it was agreed, the university medical care 



*Appendix 



13 



9 



programs must provide services that are patient-oriented and geared to the 
needs and concerns of subscribers. The organizational framework should be 
structured for patient satisfaction and, in addition, offer an appropriate setting 
for the training of future health providers and a base for evaluating the medical 
care programs. One of the key issues in the achievement of these goals by the 
medical center programs is the availability of physicians whose medical 
education and training have prepared them to work in such an organizational 
framework. This discussion led to a critical look at medical education as it is 
now structured and a review of possible changes for the future. 

A Look at Medical Education 

Medical schools have been, and, to a large extent, should be oriented 
toward research and teaching. In the past there have been limited changes 
proposed in the curricula of medical schools, and major training has taken 
place within the organizational patterns of the prestigious teaching hospital 
with its highly specialized clinics, disease orientation, and episodic treatment. 
The academic reward structure and the high standards of specialization have 
not encouraged the systematic training of physicians or paramedical personnel 
which are needed to provide primary health care. Emphasis has been placed on 
preparing and training highly skilled people who will “push back the frontiers.” 
In such a system minimal time and attention have been given to preparing 
students to meet people’s actual day-by-day health needs. 

In recent years medical schools have shown an increasing concern with 
this traditional system and with the kind of environment it provided for the 
teaching of medicine. There was growing awareness that the experiences gained 
in such a setting were not appropriate to prepare students for the rapid changes 
currently required in the delivery of health care. In meeting this challenge 
medical schools are considering new educational approaches which include the 
use of other institutional arrangements and cooperative ventures such as 
neighborhood clinics, hospital outreach centers, and other experimental 
programs. These ventures are being tested to determine what are the 
alternatives available to medical schools in their search to provide an 
educational setting in which students are prepared to meet the actual health 
needs of the community. 

One suggested alternative was a scheme whereby the basic sciences 
relevant to medicine might be taught by the university. The medical school 
would be responsible for the student’s clinical education and training. The 
clinical education and experience would be acquired in a variety of settings by 
having the student rotate through selected health service organizations 
providing care to both ambulatory and hospital patients. Such organizations as 
group practice centers, general and specialty hospitals, nursing homes, 
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rehabilitation facilities, neighborhood health centers, mental health facilities, 
etc. would be chosen to represent a whole range of experience in patient care. 
These organizations would be formally affiliated with the medical schools, and 
the schools would be responsible for planning and supervising student 
programs. As in Europe, basic research would be carried out largely in special 
institutes. Under this model of medical education, the design of appropriate 
organizational relationships between the medical school and the affiliated 
health service facilities which would provide the clinical teaching locus would 
be crucial. On the one hand, the medical school has the responsibility, and, 
therefore, must have a real voice in determining the quality of training 
provided in these facilities. On the other hand, universities per se are not well 
equipped to administer the day-by-day service facilities. 

Conference, participants took as a premise that health is a social utility 
and that basic health care is a fundamental right of the population. The task 
then becomes one of changing the framework of medical education so that 
ultimately the educational structure will be responsive to the health require- 
ments and needs of a population and to the manpower demands of an 
appropriately designed health delivery system. 

The discussions clearly emphasized the need, at the present time, for 
rather radical changes in the total structure in which health services are to be 
provided. But the group warned against thinking of changes only in terms of 
the teaching or specialty hospital. This short-sighted view could delay and 
might prevent reordering the total structure. There was agreement that some of 
the primary concerns of medical education should include: the production of 
physicians appropriately prepared for service in the newer models of medical 
care; the reaction of man to his changing environment and the consequent 
requirements for health protection; and the provision of opportunities for 
experimentation with new methods in the delivery of community and personal 
health services. 

There was concern that if the medical schools did not redirect their goals 
toward dealing with the whole cycle of life as the basic framework for health 
care, the universities will not be able to encompass a frame of reference 
essential for providing the kind of services that are needed by communities. 

Setting the Stage for the Conference 

The preliminary session provided a useful framework for conference 
discussions. In addition to a review of the structure, organization and financing 
of the plans represented at the conference, the group presented the array of 
concurrent objectives being pursued— a critical factor to be dealt with in 
discussions of evaluation. Finally, they placed the university medical group 
plans in the wider context of a rapidly changing environment in both health 
service delivery and medical education. 
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PART I: 

BACKGROUND PAPERS 

THE COLUMBIA MEDICAL PLAN 
OF THE JOHNS HOPKINS MEDICAL INSTITUTIONS 

Robert M. Heyssel, M.D. 

The Johns Hopkins-Columbia Medical Plan was developed with a series of 
broad objectives in mind, each of which had specific elements designed to: 

• develop in a new city a medical care plan based on prepayment and 
group practice for all citizens of the city and surrounding area who 
elected to join. This meant developing a means of marketing which 
made it possible for all residents to join irrespective of their 
participation in employment based groups. 

• assure all potential members a true dual choice in medical care by 
encouraging other non-plan health providers in the area and, where 
practical, developing working relationships with them. The University 
intends to adopt a new stance toward private practitioners in the area 
and actually encourage the development of other modes of practice in 
order that a practical atmosphere of real competition between a 
prepayment plan and the usual fee-for-service option available to most 
people will be present. 

• develop a plan that would be replicable as an economic health service 
package in other areas of the country. To do so the plan could not be 
heavily subsidized by funds other than member payments although it 
was recognized that an investment in start-up and development costs 
would be necessary. It was planned that this investment would be 
recovered over a long term of years-thirty or more-by small add-ons 
to member premiums. 

• enlarge the concept of the Columbia Medical Plan to include 
community health services and education, as well as personal health 
services. 

• assure that the plan was developed in such a manner that the 
University’s interest in teaching and research would not be com- 
promised, although this interest would be developed within the limits 
of practicality. The possibility had to be preserved for developing 
arrangements for different levels of deductibles and co-payments, 
differing benefits and service structures, the possibility of different 
modules of patient care, and the possibility of designing into the 
programs different uses of new or old health personnel. 
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The realization of all of these University objectives required that the 
medical services be delivered in a manner which was acceptable to people, 
which assured quality of service for people, that was responsive to the people’s 
needs as they perceive them and that services were delivered at a cost which is 
reasonable. These are not subordinate clauses to the specific objectives outlined 
before, but rather reflect two facts. The first is as stated, the specific objectives 
can’t be realized without first satisfactorily meeting the expectations of 
members enrolled in the plan. Secondly, there are in existence plans which 
satisfactorily demonstrate the acceptance of many people of prepaid group 
practice. University involvement planned to meet the University’s commit- 
ments to teaching and research, society’s needs for new patterns of health 
services delivery, and the consumer’s personal interest in the quality, 
availability and cost of medical care, is unique and should reconcile these 
various and, perhaps, conflicting objectives. 



MEMBERSHIP AND DATA COLLECTION 

The new City of Columbia and vicinity appears to offer great potential in 
relationship to these objectives. In the first place it is a specifically designated 
geographic area. As a new city, it is planned to grow in a reasonably controlled 
manner from a very small number of people who already reside there to a 
population of over 100,000 in ten years, allowing the plan to develop slowly 
along with the community. Moreover, the new city is to be mainly 
middle-class, although it was envisioned that some upper middle-class persons 
would reside there and low cost housing is planned as part of the city. 

The current population of Columbia of 6,500 is slightly above the medium 
national income. It is a younger population than the national average although 
increasing numbers of older people are moving into the city. Since all members 
of the Columbia Association, which means any resident of Columbia, may join 
the plan on a group basis, either with full membership coverage or with a 
supplemental coverage to their existing health insurance, the total percent of 
the population which selects the plan, in the final analysis, depends upon the 
attractiveness of the offering and the acceptability of services. 

One of the obvior.s uncontrolled factors is that most people have existing 
employer provided health insurance. Since it may not be economically 
desirable for such an individual to change his coverage because he might lose 
his employer’s contribution to his health insurance coverage, a supplemental 
plan was developed for Columbia residents which will decrease in importance 
as marketing of the plan by carriers improves. The population has no other 
choice of a prepayment health plan and, at this time, because of limited 
numbers of doctors and limited hospital beds available in the area, non-prepaid 
sources of care are inadequate. Currently, there is no individual enrollment in 
the plan. 
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Data obtained by the carrier at time of enrollment is basically name, age, 
sex, place of residence. On enrollment, each individual member is sent a very 
short questionnaire (Figure 1). Based on the replies to this questionnaire 
priorities for an appointment for a health review by a plan physician are set. 

Enrollment data are filed by family index number and individual 
members are assigned a medical history number. A representative sample can 
be readily drawn by family or individual histories. The insurance carrier 
notifies the plan at the time they receive notices of enrollment and similarly 
when cancellations or changes in membership occur. 

Finally, a master sample of the total population for Columbia and the 
surrounding area compared to the enrolled population is in preparation. 

MEDICAL CARE PROCESSES 

The system is set up with an F.ncounter Form which is the basic 
instrument for recording what services are provided to each patient during a 
specific patient encounter. This form has been in use since the beginning of 
operation. An analysis of these data makes it possible to tell the number of 
physician services per any patient or family unit, which physicians saw the 
patients, where they were seen, whether the services were problem oriented or 
whether they were not, referral patterns, etc. It is possible to tell the kind of 
laboratory or X-ray work, type of service, place of service, type of test and 
procedures. The number of hospital admissions will be monitored closely 
including the length of patient stay and other characteristics since this is so 
crucial to financial operation of the plan. 

Because visits to the physician are classified as either problem, that is 
prompted by a medical complaint of the patient, or as non-problem oriented, it 
can generally be assumed that non-problem oriented visits are for preventive 
services. The data are tabulated on a monthly basis for management purposes, 
and are also available for research interest. There are plans for comparisons of 
the data with other populations than that of Columbia City; for instance, East 
Baltimore. Moreover, if cooperation of doctors who are practicing in the area is 
forthcoming, we would hope to be. able to provide a comparison between the 
prepayment enrolled population as opposed to the population who are seeing 
physicians on a fee-for-service basis. This should be possible through the master 
demographic study we intend to compile. 

It is planned to measure behavioral aspects, attitudes, and satisfactions of 
members and the staff. This aspect of the plan has not been discussed to any 
extent to date. However, very recently the first member meeting was held and 
a group of ten people was formed as an ad hoc committee to develop the 
mechanisms for selecting and outlining the duties of a Member Advisory 
Committee drawn from the membership. This group of ten was randomly 
selected from fifty who indicated a desire to work on the problem. They will 
work with the administrative officers of the plan in defining the duties of the 
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Advisory Council. The Board of Columbia Hospital and Clinics Foundation had 
previously agreed ; ; accept a consumer member on its Board if the Advisory 
Committee and the membership so desired. 



MEDICAL RECORDS AND PATIENT CARE RESEARCH 
AT COLUMBIA 

The medical records system at Columbia Hospital and Clinics Foundation 
is problem oriented. The question of patient compliance with the therapeutic 
regimen has no built-in follow-up mechanism at the moment other than that 
relating to the physician-patient relationship. Some key information probably 
is not in the chart, such as patient satisfaction or dissatisfaction with services 
received. 

The Columbia Medical Plan is an operational unit of the Office of Health 
Care Programs which has within it a Health Services Research and Development 
Center devoted to specific research in the delivery of medical care. The Health 
Services Research Center is charged with not only the evaluation of programs 
through a central core staff but for aid in development of programs. In order to 
be sure that research interests are protected, certain personnel in Columbia as 
well as other operational arms of the Office of Health Care Programs are 
members of the staff of the Health Services Research Center as well as the 
operational arm in which they have line or staff functions. This enables them 
to have an overview of how the program is going and also be able to monitor 
changes in the program so that there is a feedback mechanism to the Health 
Services Research and Development Center. It is the hope that this mechanism 
will develop specific research questions and methods of obtaining answers. 



ORGANIZATION AND MANAGEMENT OF THE 
COLUMBIA MEDICAL PLAN 

The Office of Health Care Programs is a unit which is a joint venture of 
The Johns Hopkins University School of Medicine and the School of Public 
Health and Hygiene and The Johns Hopkins Hospital. Joint appointments are 
obtained for its personnel in appropriate divisions in those institutions. The 
Medical Plan is linked with the teaching program of the University. Students 
are involved in some of the programs, particularly in Pediatrics and Medicine, 
at the Columbia facility and subsequently it is envisioned that resident staff 
will also be involved. The staff of the Columbia Medical Plan have faculty and 
staff privileges at Johns Hopkins Hospital and as such participate in programs 
in East Baltimore as well as in Columbia. 
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Financing of the teaching program at this stage has not been settled; 
however, we would desire to fund separately and account for teaching, 
research, and services. An attempt has been made, and is being made, to obtain 
accurate cost for providing each service since, in a multiple carrier system in 
which variable benefits will occur, it is necessary to have cost by service 
rendered in order to maintain an equitable capitation rate amongst carriers and 
to provide data on cost effectiveness. The cost accounting system does provide 
a method for obtaining estimated or actual start-up costs. As an example, the 
budgeted per capita cost for the projected number of members for fiscal year 
1969-70 is $165. This includes $115 per capita based on a balanced financial 
operation. The difference is considered to be developmental costs related 
primarily to initial overstaffing required to provide a wide range of services to a 
limited number of subscribers. To this point, there have been no monies 
donated. Capital for construction and equipment is totally privately financed 
to be amortized over a period of years. 

CONCLUSION 

The foregoing description is a broad outline of the Columbia Medical 
Plan which is a prepaid, comprehensive medical service plan, totally privately 
financed. It is intended to be a focus for research and teaching as well as 
service. The plan is in process of development with many areas as yet 
incomplete in detail. 
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Figure 1 

NEW ENROLLEE QUESTIONNAIRE 

Please check and/or describe as appropriate: 

I. Do you have a current complaint about your health? 

No 

Yes If so, please describe 



II. Do you have a history of a medical problem which needs current 
evaluation? 

No 

Yes If so, please describe 



III. Are you currently taking any drugs prescribed by a physician for the 
treatment of a complaint or disease? 

No 

Yes If so, please give drug name, if known, and prescription 

number and name of pharmacy from the prescription label. 



IV. Is there anything else which you want to discuss with a physician at this 
time? 

No 

Yes If so, please describe _ 
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RESEARCH AND EVALUATION IN AN OEO 
NEIGHBORHOOD HEALTH CENTER 



COMMUNITY GROUP HEALTH FOUNDATION, INC. 
Washington, D.C. 



James D. Shepperd, M.D. 
Edna K. Haedke 
Paul B. Comely, M.D. 
James D. Steen 



The long range objectives of the Center were designed to: 

• intervene in the poverty cycle through improving the health conditions 
of the residents of the Upper Cardozo community by providing better 
health care. 

• improve income and manpower development by providing jobs in 
health service related fields that afford upward mobility. 

• promote change in local and national institutional patterns of 
delivering health care, and developing health manpower by demon- 
strating hew and improved approaches. 

The immediate goals of the Center are defined and identified as follows: 
comprehensive, coordinated, and continuous services provided by the Center 
program should include preventive health services, diagnostic service, treat- 
ment, family planning, comprehensive in-home care of die chronically ill and 
other home health services, rehabilitation services, dental care, mental health 
services. 

The Center must be accessible to residents and deal effectively with 
barriers to medical care usually encountered by the poor, including costs, 
distance, transportation, language, hours during which services are available, 
and cumbersome intake and eligibility procedures. 

Health services must be acceptable and accessible to patient and must 
provide mechanisms to eliminate the problems that have so frequently plagued 
other health service institutions for the poor, including unpleasant physical 
surroundings, crowding, long waits, and depersonalization. 

Health services must meet high standards of quality, be under competent 
professional supervision, and include continuing and effective quality controls. 
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The care should be family oriented and all members of a family should 
be treated by the same physician, to the extent that the physician or team of 
physicians are trained to provide the needed care. 

The responsibility of physician for patient’s care should continue when 
the patient is transferred to a specialized professional or health facility and 
between clinical episodes. 

The Center should provide employment and training opportunities to 
attract and train residents from the community to perform jobs in the health 
services field and offer placement in positions offering upward mobility. 

Center patients’ services are to be coordinated with all other community 
sources of health services, and with other non-health resources which serve the 
neighborhood such as schools, churches, social agencies, etc., with arrange- 
ments made for cross-referrals and exchange of service. 



TARGET POPULATION 

Preliminary base-line data developed by National Opinion Research 
Corporation (NORC) in a survey of. the seven census tracts in the Upper 
Cardozo Area projected a total population of 45 ,000 within the target area. Of 
this figure some 15 percent or about 7,000 persons, representing the eligible 
residents of the community, will be served by the Center in its initial attempt 
to increase the availability of medical services to this area. This target 
population represents families who are eligible under the OEO poverty 
guidelines, the local Medicaid guidelines, or the Medicare program. 

The principal reasons for selecting this specific area were: community 
initiation of the project starting back in the early 1960s and continuing 
community support ever since; socioeconomic mix; lack of access to adequate 
facilities by the target population; high incidence of health and social 
problems. 

While these and many other reasons must be viewed as elements in 
establishing a rationale for this Center, a somewhat closer scrutiny of available 
data points out significant factors used in determining the location. 



DEMOGRAPHIC FACTORS 

Upper Cardozo is a multi-faceted and diverse community. According to 
preliminary data resulting from the NORC survey, the mean income of the 
community is $5,800 with extreme distributions in range. Forty-one percent of 
the families have a pre-tax income of less than $4,000 per annum, while 16 
percent of the families earn over $10,000 a year. Seven percent of the 
population is on welfare. Four of the seven census tracts contain a 
disproportionate number of poverty or near poverty families. Ethnically, the 
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community is 79 percent Negro- American; 12 percent Caucasian; and eight 
percent Latin and Caribbean American, most of whom are Spanish speaking. 

The median age of the population is estimated at 27 with 25 percent of 
the population between the ages of one to 13, and seven percent over 65. The 
family’s average size is 2.7 persons with 32 percent being single member units. 

The median completed grade of schooling of the head of household is 
grade eleven. Thirty-three percent of the households are headed by females. 

The median residence in the neighborhood is four and a half years with 
19 percent of the families living there less than one year while 30 percent have 
been in the area for ten or more years. 

The preliminary NORC survey information does not provide detailed 
health status and utilization data for the community, the specific demographic 
characteristics of our target population of approximately 7,000 people, or 
correlate demographic characteristics by census tract. These kinds of analyses 
will be possible when the final information is received. At that time, it will also 
be possible to correlate the NORC findings to other recent studies of the Upper 
Cardozo Area, including the Mayor's Economic Study, the Redevelopment 
Ford Agency studies of Metropolitan Washington, and other studies conducted 
by the United Planning Organization (the District of Columbia CAP agency), 
and the District Government. 

In the meantime, certain things are clear. Based on 1967 Health 
Department figures, using such traditional health indicators as infant mortality, 
tuberculosis, venereal diseases rates, etc., the Upper Cardozo area is among the 
highest in the city. In addition, it can be inferred from the data available that 
the poorest, the largest, the least educated, and the least healthy Negro families 
are disproportionately concentrated in four of the seven census tracts in the 
area. 



ELIGIBILITY AND ENROLLMENT 

There are five categories of patients that are seen by the Center: eligible, 
ineligible emergency, Medicaid, outside the target area, and employees. Eligible 
families must (1) live in the target area boundaries; (2) have a Medicaid or 
Medicare card or have a family income that falls within the OEO poverty or 
District of Columbia Medicaid income guidelines. Both of these guidelines 
relate family size and family income and are quite close in terms of income 
allowance. If a family lives outside the area, or does not have a Medicaid or 
Medicare Card or meet prescribed income guidelines, that family is classified as 
ineligible. 

To save the patient time and the humiliation of eligibility checks, the 
registration takes the word of the registrant on address and income, and no 
follow-up investigations are made by staff. If a family is eligible for Medicaid, 
but not currently enrolled, an effort is made to start application procedure. 
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The eligible family is entitled to full comprehensive, family-oriented care. 

An ineligible patient is treated only on a one-time emergency basis and is 
referred to another institution for any follow-up care. According to regulations 
of the District of Columbia’s Medicaid Program any agency that accepts 
Medicaid funds must treat all eligible District of Columbia Medicaid patients, 
no matter where they reside in the city. Since the Center is heavily dependent 
on Medicaid reimbursements, it treats those Medicaid patients from outside the 
area on an individual, episodic basis almost exclusively. 

The Center employees are given general screening exams and are treated 
for episodic health problems that arise. 

The enrollment process is family -oriented with the head of the household 
dr a responsible member of the household completing a Family Registration 
Form, which collects such demographic information as tenancy status, census 
tract, number of related and unrelated household members, family income, and 
welfare status. One copy is distributed to the Registration Desk, one is 
attached to the head-of-household’s Medicaid chart, and the other is sent to 
Bio-Dynamics, Inc., for data processing. 

An Individual Registration Form is made out on an individual family 
member when he comes into the Center for treatment. This form, which 
collects information on age, sex, marital status, race, ethnic group, employ- 
ment, and education is attached to the patient medical chart,, and copies are 
sent to accounting for billing purposes, and Bio-Dynamics for statistical 
reporting. 

All patient data, including medical records and registration information 
are filed by a seven digit Family number. The first five digits are the Family 
designation which is categorized so as to make a ready distinction between 
eligible, ineligible-emergency, Medicaid outside the area, and employee 
patients. The last two digits are the Individual designation which indicates his 
relationship to the head of household. 

A representative sample of the population registered could easily be 
drawn from the entire numbering system or any of the specific categories. A 
block of six hundred eligible, pre-registered patients already has been set aside 
for possible follow-up study. 

Currently there is no plan for systematically reviewing patient eligibility 
or enrollment. However, as the staff finds changes in address and Medicaid 
status affecting an individual’s eligibility in the program, this information is 
routinely supplied to the registration desk, medical records, accounting and 
Bio-Dynamics. 



MEDICAL CARE PROCESS-UTILIZATION 



The Center is presently installing a computerized information system 
with the help of Bio-Dynamics, which will collect information on the 
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characteristics of the patients utilizing the Center and the services being 
provided to these patients by the Center staff. This information will be utilized 
for management evaluation and reporting, billing and research purposes. 

At present, the Center is using standardized pre-coded forms to collect 
service information on physician’s encounters, dental encounters and lab 
orders. The physician’s encounter form, used by all the medical doctors on the 
staff, collects data on the date of the encounter, type of provider, service 
location, type of encounter, length of encounter, services rendered and 
diagnosis. The dental encounter form, used by all dentists and dental 
technicians, obtains the same information on dental services minus diagnosis. 
The lab form indicates the date ordered, the provider who requests test, the 
location of the processor, and the specific test ordered. 

To provide utilization data, information from the above forms and from 
patient registration forms are used in an attempt to correlate the frequency of 
encounters, frequency of patient visits and patient characteristic variables. 

The Center is presently designing forms to monitor other professional 
encounters, including services provided exclusively by nurses, social workers, 
family health workers and x-ray technicians in the Center. The record of home 
visit encounters will be used to monitor services provided by all staff in the 
home, and External Referral information will be used to measure direct and 
indirect health service referrals made by the staff to outside agencies. The 
External Referral Form will indicate number of admissions to extended care 
facilities and hospitals, including information on number of in-patient days, 
services provided and number of live and still births. 

As a Comprehensive Neighborhood Health Center, the provision of 
preventive services is a very important feature of the program. For the purposes 
of designating types of physician encounters, the staff attempts to distinguish 
between episodic care, long term health care management, initial health 
assessment and routine or preventive care. Routine and preventive care are 
defined as including encounters for pre-natal care, postpartum care, pregnan- 
cies without complications, routine physical examinations (including family 
planning guidance), preventive immunizations, health education and coun- 
selling, well child and well person care. Although prevention is an important 
feature of the program, the analysis of data collected in the first month of the 
Center’s operation indicated that use of preventive services was minimal. 

In connection with dental encounters the staff attempts to distinguish 
between diagnostic, preventive and therapeutic services. Preventive services ce 
defined as routine and preventive prophylaxis, topical fluoride application, 
structural dental health education, and other preventive services such as 
protective mouthguards and diet analysis. 

The patient registration and .service utilization information will be 
tabulated at hast monthly. The Center eventually plans to use this information 
to produce a billing print-out as well as statistical reports. Similar information 
systems, although not quite as extensive, are being installed in the various OEO 
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Neighborhood Health Centers across the country. When these installations are 
further developed there will be the opportunity for comparison on certain 
utilization indices with other Neighborhood Health Centers having similar 
populations and programs. 



There are a number of crude indications in our present record system 
which might be used in measuring patient attitudes and behavior. 

In the Family Registration process, each head of household is asked 
about who referred him to the Center. Changes in referral patterns may 
indicate changes in patient acceptance, health education efforts and outreach 
activities. 

The experience of other Neighborhood Health Centers indicates that the 
broken appointment rate runs extremely high. The Center’s appointment form 
provides for a regular tabulation of data on appointments cancelled and 
no-shows by type of health care provider. Follow-up studies may be done on 
the above data to ascertain the reasons for them, including patient dissatis- 
faction. ‘This appointment information could be correlated to the patient 
characteristics and to Center’s registration data which is regularly collected in 
the registration process. 

Another possible tool for measuring patient behavior is the Center’s 
External Referral Form which is currently being developed. Among other 
things, this form will enable tabulations on the number of Center-initiated 
referrals for which the patient did not follow through. 

A short questionnaire study is also being considered to determine patient 
acceptance of the services provided by the Center. The method of analysis 
might include a short interview of a small group of registrants, perhaps our 
pre-registered population, comparing the attitudes of a sample of registrants 
who regularly utilize service and those who do not. 

One of the main objectives of the Center is participation of the 
community in the operation of the Center. There is an Advisory Council, 
whose membership will eventually include a majority of persons who are 
eligible for care in the Center. In addition, one-third of the Board Members are 
representatives of the local Community Action Agency and are considered 
quasi-community spokesmen. A great number of the employees, including all 
trainees, are residents of the target area. 

No specific measures have yet been developed for measuring progress 
toward this objective. However, among the dimensions that the Center will 
want to consider are: (1) To what extent is the community represented? (2) 
How much participation is there? (3) What effect has the community 
participation had at its impact points, including the Advisory Council, the 
Board, the staff, community organizations and residents? (4) How adequate is 
the process of community participation used? 



ATTITUDES AND BEHAVIOR 
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There are no plans at the moment to formally study staff attitudes and 
behavior. However, the Center does have a one-year technical assistance 
contract with the University Research Corporation (URC) of Washington, D.C., 
to help the staff better analyze and define professional and para-professional 
roles and responsibilities in the Center and begin to develop realistic career 
ladders for para-professional staff. As an off-shoot of URC’s work with the 
Center, there is much information feedback on staff morale, satisfaction and 
behavior. 



RESEARCH AND EVALUATION 

The Research and Evaluation Department of the Foundation will be 
devoted to coordinating the specific research projects rotated to the delivery of 
health care. An on-going evaluation of the medical care process will enable the 
administration to improve the decision-making capability by providing accurate 
information and analysis. Research will be conducted by all divisions of the 
Health Center structured by the Research Division. 

The staffing patterns will be as follows: Director of Research, Deputy 

Director, Information Retrieval Specialist, and Secretary. Additional personnel 
may be assigned during collaborative studies, and consultants are available. The 
Research Department will be under the auspices of the Health Center. In 
addition, it will work closely with Howard University Department of 
Community Health Practice. 



TEACHING 
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A major effort is being made to bring disadvantaged community residents 
into the health service work force. The success of this effort will be measured 
in terms of numbers trained and employed as well as how effective the 
residents become as members of the health care delivery team. The training 
categories are family health workers, dental assistants, dental x-ray technicians, 
dental laboratory technicians, medical laboratory technicians, medical x-ray 
technicians, health educator aides, medical records aides, pharmacist aides and 
physician assistants. This teaching program is operated by the Center in 
cooperation with Howard University and Freedmen’s Hospital as well as other 
agencies. 

Staff in-service training will serve to improve the skills of all employees. 
Staff members will be offered on-site training and encouraged to take 
advantage of “release time.” Courses are offered at Howard University, 
Washington Technical Institute, as well as other organizations which are able to 
offer certification in whatever field of study has been completed. 
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The Center serves as a training site for a variety of students enrolled at 
Howard University schools. The medical complex including the schools of 
medicine, nursing, dentistry, pharmacy and nutrition have plans to include the 
Center in the field service curriculum. 

Resident physicians and dentists will participate in direct services to 
patients as well as community activities. These physicians will be oriented to 
the working of our group practice plan. Howard University Department of 
Medical and Dental Community Health Practice will be primarily responsible 
for these aspects of the program. Faculty supervision of the undergraduate and 
post graduate programs is expected. 

The evaluation staff of OEO plans collaborative studies of all neighbor- 
hood health centers in the field of manpower. This study will indicate patient 
staff woikload ratios, staffing characteristics, staff satisfaction and aspirations, 
salary comparability as well as relationship between “Center trained” staff and 
“professionals.” 



OUTCOME MEASURES 

The major focus of the Research and Evaluation Department will be to 
develop appropriate and reliable measures of several outcomes. This is glib 
expression of the desire to measure several aspects of a center's impact on the 
community; however, these studies are exceedingly difficult to develop so that 
the measurement reliability supports the contention made by the investigation. 

We have selected four areas for study within our organization and will 
participate in collaborative studies with Howard University Center for 
Community Services, the National Medical Association Foundation and the 
District of Columbia Health Department. We have also been requested to 
participate in studies conducted by Tufts University, the Washington Heart 
Association and the National Academy of Sciences. 

The collaborative study to be conducted by the Howard University 
Center for Community Services will measure sociological as well as health 
aspects of the neighborhood health center in Shaw and Cardozo areas in 
Washington, D.C. This study will include our registered population, that of the 
adjacent area without a neighborhood health center, and another area which 
has a 314-E project called a “Group Practice Without Walls.” The studies 
include epidemiological evaluation of such tracer Health States as perinatal 
mortality, TBC rate, otitis media, dental caries and hypertension; utilization 
patterns as reflected in survey studies to be performed by the District of 
Columbia Health Department; consumer satisfaction as indicated by in-house 
surveys at each establishment; cost analysis; health care process; and citizen 
participation. 

The Center Research and Evaluation Department will perform in-house 
evaluations as indicated in earlier sections. A research project to determine the 
efficiency of the health care plan is being developed, based on the following 



hypotheses: 1) that paramedical personnel can extend the efficiency of scarce 
physician and dentist staff; 2) that compliance with the provider’s instruction 
increases the efficiency of the delivery system. 

The method will include a comparison of the two family-care teams’ 
ability to deliver care, using one team of nurses and paramedical personnel to 
re-enforce all physician instruction, and the other without this additional 
device. In the former case, re-inforcement will take the form of consultation 
with a nurse and/or family health worker after each physician visit. A home 
visit will-be made to see if there has been complete use of all possible resources. 
The second team will omit the consultation and follow-up visit and use 
resources only upon patient’s request. 

The team e valuation will be considered in terms of achievement of health 
care goals. These are examples: control of major disease problem, using 
biological measurements with diabetes serving as one index disease; reduction 
in disability days; compliance with physician instruction; patient compre- 
hension of his disease; patient attitude toward delivery system as indicated by 
questionnaire; number of visits needed to achieve a biological outcome, for 
example, control of hypertension. 



IMPACT ON OBJECTIVES 

The Center is presently trying to design measures to show how well the 
program is meeting its overall objectives and immediate goals. Examples of 
specific measures are given: 



Comprehensive Care 

Goals 

To immunize children before 
they attend school against 
dipthei.a, pertussis, tetanus, 
polio and measles. 

To obtain and have recorded 
by the third visit a complete 
medical history and physical 
examination of all children 
and adults attending Center clinics. 

To detect tuberculosis and 
effectively treat cases and 
recent converters. 



Methods 

Audit charts to see if immunizations 
are recorded and given: immuniza- 
tion survey in community or among 
mothers bringing their children to 
the clinic. 

Chart audit. 



Chart audit for tuberculin-testing and 
treatment of recent converters; stud- 
ies of compliance with medication 
using urine testing for INH. 
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Goals 



Methods 



To detect early carcinoma 
of the cervix or pre- 
malignant lesions. 

To deliver adequate home 
services. 



Coordinated Services 

To insure that communication 
and referral to back-up 
hospital occurs. 



Continuous Care 

To insure continuity of care. 



To determine if medical care 
is acceptable to the 
population. 



Audit charts for obtaining and re- 
cording Pap smears. 

Chart audit of patients eligible for 
home services to determine if they 
have been received; interview pa- 
tients receiving home service to 
determine attitude toward this 
service. 



Chart audit to determine lag period 
from time of referral to hospital until 
return; completeness of communi- 
cations and also follow-up pro- 
cedures. 



Chart audit to determine number of 
physicians, hospitals or other pro- 
viders involved in the care of a single 
patient over a period of time. 

Calculation of broken appointment 
rate; survey of attitudes. 



COSTS 



At the present time the Center has no cost accounting system other than 
the traditional object classification system. It is considering the adoption of a 
functional cost system currently being developed under the auspices of OEO. 
This system would distribute costs on an accrual basis, including detailed 
breakouts for services, training and research which would be aggregated 
quarterly. Utilizing this system with an ongoing information system, it would 
be possible to begin developing unit cost data. The OEO system is visualized as 
a national reporting system which would not allow comparison between all 
neighborhood health centers, but would allow comparisons between other 
centers with similar clinics and programs. 
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EVALUATION OF HEALTH CARE DELIVERY 
IN THE COMMUNITY HEALTH CARE CENTER PLAN 
NEW HAVEN, CONNECTICUT 



G. K. MacLeod, M.D. 



The general objectives of the Community Health Care Center Plan in 
New Haven are to provide prepaid, comprehensive family health care through a 
medical group practice within a community -sponsored program, and to 
collaborate in the training of personnel for the group practice of medicine and 
in the performance of research and evaluation studies— through an affiliation 
with the Yale-New Haven Medical Center. The major objectives of the Plan are 
to: 

• provide comprehensive family health care through a group practice 
mechanism, emphasizing the importance of continuity and coordi- 
nation of services in preservation of well-being, in prevention of ill 
health and in treatment of disease; promote safeguards of high quality 
while seeking economies in the utilization of skilled manpower, 
inpatent facilities and specialty resources; meet the fiscal needs 
through contractual prepayments and cost reimbursements; and 
achieve satisfactions for both the population served and the providers 
of service. 

• experiment on the design of group practice teams for effective 
utilization and training of personnel; train next -generation physicians 
and nursing practitioners for effective participation in a group practice 
setting; and train personnel for medical care administration, health 
education and community service. 

• perform health services research through pre- and post-enrollment 
studies; evaluate service and training objectives through ongoing 
studies; and, eventually, undertake special administrative and clinical 
epidemiological studies. 

EVALUATION 

Current design and development projections provide for studies and 
evaluation of accumulating experiences within the service and training 
programs of this demonstration project. A partial list of proposed topics for 
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study includes the incidence and prevalence of illness, utilization of medical 
care services, patterns in the need and receipt of care, and attitudes toward 
illness and medical care-all in relation to social, economic, cultural and 
demographic characteristics of the patient population. We also propose to 
undertake studies on the economies of the pre-payment mechanism; studies or. 
the effective use of provider personnel; studies among the enrolled population 
on the actual utilization of the group practice and of outside services; 
comparative studies on illness, the receipt of medical care, costs, etc. among 
enrolled and nonenrolled populations in the community; studies on the 
measurement of quality of care; and eventually, studies of end results. The 
studies should extend to observations to identify problems in the start-up and 
in the early years of operation; to learn the consequences of enrollment on a 
“dual choice” (voluntary) basis; to explore health center design and inter- 
relations between a community-sponsored comprehensive health care program 
and a teaching medical center; to project expanded programs for dental and 
psychiatric services beyond the limited provisions proposed at the outset; and 
to forecast the potential need for an automated multiphasic health screening 
program. 

The Health Care Plan is intended to be experimental with respect to the 
structuic of the group practice team itself; to explore, test and evaluate 
alternative patterns in organization and staffing for comprehensive family 
health care; and, whenever possible, to upgrade the functioning of physicians 
and nurses on the team. The initial number of subscribers will determine the 
number of alternative experimental group practice “modules.” With small 
enrollment, only a single pattern may be feasible, leaving further elements of 
design to be introduced and tested at a somewhat later date. If the initial 
enrollment is large, two or more designs may be implemented at the outset. 
Studies and evaluation will also be performed on methodology and produc- 
tivity in the training of professional, administrative, technical and supporting 
personnel for participation in comprehensive continuing care through prepaid 
group practice. 

For purposes of research and evaluation, the Plan intends to enlist the 
support of the Yale-New Haven Medical Center, in the form of faculty and 
cooperating personnel from the Department of Epidemiology and Public 
Health (DEPH) and from other departments of the Medical Center. 

Since the New Haven Plan is still pre-operational, our present focus is 
primarily on studies of population groups, design of expected operational 
procedures and preparation for evaluation of future performance. Such division 
into categories is not intended, however, to segregate areas which should be 
interrelated. The first steps, now underway, include pre-enrollment population 
studies to compile benchmark data in advance of becoming operational, and 
the design of a system for recording and retrieving clinical and administrative 
information. 
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Opportunity exists to coordinate preliminary research activities with 
other community or university-sponsored comprehensive health care plans 
which, by present perspectives, will utilize the resources of the Yale-New 
Haven Medical Center for certain specialty services and inpatient hospital care 
in much the same fashion as the Community Health Care Center Plan. 

TARGET POPULATION 

The New Haven Standard Metropolitan Statistical Area has about 
350,000 people, with about 140,000 in the city itself. Most medical care in the 
Greater New Haven area is provided by physicians in fee-for-service solo 
.practice, using two large teaching general hospitals. Both of these hospitals 
have busy emergency rooms whose utilization has approximately doubled over 
the past ten years, much of it nonemergent. Outpatient clinics in both hospitals 
function mainly as teaching facilities and are oriented toward subspecialty care; 
they are not set up to provide continuing comprehensive care. In addition, 
there are two neighborhood health centers in New Haven-primarily for 
medically indigent children-in ghetto areas immediately adjacent to each 
hospital. Although neither program charges for its services at present, one of 
them is investigating the possibility of developing a prepayment mechanism. 

Primarily because of the research and demonstration implications of this 
project, the Plan has been designed to care for a population representing, more 
or less, a cross section of the Greater New Haven area. In part because of a 
pre-judgment of the desirable level for a single comprehensive group practice 
unit and in part because of the tie-in with the Medical Center, our target 
population has been set at about 30,000. 

The Plan has historical ties with the labor movement of the Greater New 
Haven area and was strongly supported in its initial planning and development 
efforts by the union leaders and memberships. Consequently, the population to 
be served will, in part, be labor union members and their dependents— primarily 
employees who have achieved dual choice provisions through negotiated 
contracts. 

In attempting to achieve a cross sectional representation, enrollment will 
also be open to professional, management and other nonunionized groups who 
are able to provide for participation of their memberships on a voluntary basis. 
In addition, we plan to include up to 20% of the existing enrollment from the 
medically indigent and public assistance populations. As far as possible, 
nongroup (individual) enrollment will be avoided-at least during the initial 
stages— because of the risk of large adverse selection. 

Preliminary activities to achieve the enrollment of the target population 
for the Community Health Care Center Plan have been organized around a 
small full-time staff, working with the health education committee of the 
Central Labor Council which represents nearly all labor unions in the area. A 
subcommittee of the Community Health Care Center Plan Board of Directors is 
also concerned with enrollment. 
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Pre-enrollment studies are now underway, with special emphasis on those 
groups which already have dual choice provisions and whose memberships will 
be eligible to enroll in the Community Plan. Similar pre-enrollment studies will 
soon be conducted on the welfare population. The intention is to conduct 
subsequent parallel studies among those who do and those who do not enroll in 
the Plan— for future comparisons with pre-enrollment benchmark data.- 

Specific data to be collected after actual enrollment begins have not yet 
been determined. It is intended, however, to compile comprehensive informa- 
tion about enrollees, extending to demographic and bio-medical histories for 
individuals and families. 



MEDICAL CARE PROCESSES 

Utilization: We intend that our functional performance and record 
system shall be sufficiently sophisticated to provide for the storage and 
retrieval of information on the characteristics of the target population, on the 
services provided (to whom, by whom, where, why, etc.), on the progress and 
outcome of medical care episodes, and on allocated costs. In other words, we 
hope to have-more or less routinely— a considerable body of data on the entire 
service program. Thus, analyses along these lines should lead to data on the 
care provided in each individual case, on the care in each episode of illness, on 
care in cases with related or unrelated multiple conditions, on care received by 
individuals or families over a period of time, and on the composite of all care 
provided in the Health Care Plan. 

The number of physician services by diagnosis, by unit of time, by type 
of physician, by place of service, by patient characteristics and by frequencies 
among enrollees will be within the capacity of our research and evaluation 
undertaking. 

Similarly, the number of services by health professionals other than 
physicians, the number of hospital admissions as characterized by length of 
stay and diagnosis, the number of admissions to extended care facilities and 
other inpatient facilities, and the number of preventive techniques are all 
within the framework of our proposed administrative studies and our research 
and evaluation techniques. 

Of particular importance will be the breakdown of the various categories 
of enrollees (for example, union members, other employees, welfare recip- 
ients). We should uncover-through periodic routine reports-special needs for 
administrative and clinical purposes. A specific time-table of the frequency of 
tabulation of data is difficult to make. However, we expect to provide for 
j continuing collection and frequent analysis of data relevant to the daily 

or routine operations of the Plan while compiling and analyzing data at such 
frequency as will be indicated for information which can be batched for special 
studies. 
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In addition to the aforementioned considerations, we will also have to be 
alert to the importance of designing our data collections for comparability with 
programs such as the Connecticut Utilization and Patient Information 
Statistical System (C.U.P.I.S.S.) which is to be a statewide system developed 
under the 'auspices of the Connecticut Regional Medical Program for the 
routine assembly of patient care information; for special conferences such as 
this one; for participation in cooperative studies under the auspices of the 
National Center for Health Services Research and Development or the Group 
Health Association of America; and for comparisons with nonenrolled 
populations locally and with national data. Efforts to coordinate data with 
other developing prepaid group practice plans in New Haven will be facilitated 
through the Committee on Community Health Services at Yale. 

Attitudes and Behavior: Since the Community Health Care Center Plan 
was designed to have a Board of Directors representing various sectors of the 
community, including consumers, it is obvious that their participation will be 
present at the policy level. 

We intend to undertake periodic studies on the attitudes and the 
expressed satisfaction of the enrollees for correlation with their behavioral 
patterns, extending to and including the degree of patient compliance with 
prescribed regimens. Specific studies— such as those on the number of broken 
appointments, on the proportion of patients with repeated records of broken 
appointments, on physicians whose patients have high rates of broken 
appointments, and on the number and characteristics of enrollees who have not 
used services-represent information which would also be readily retrievable 
from the record system now in preparation. 

Because of the expected accumulation of social, economic, geographical 
and other data from pre-enrollment studies, correlation with information 
relating to availability and acceptability of services, waiting time and amenities 
will be feasible at regular intervals through the follow-up mechanism of 
post-enrollment studies. In addition, corresponding comparative studies are 
contemplated-as far as may be feasible— among non-enrollees in the com- 
munity having other insurance or no insurance. 

Research: The Plan design includes provision for a research unit and 
staff. Much of the previously mentioned research and evaluation will be 
pursued wholly by the Health Center research staff in collaboration with 
administrative and clinical staff, but some will be undertaken in collaboration 
with faculty and graduate students from the Department of Epidemiology and 
Public Health (Section on Health Services Administration and Section on 
Health Education), from the Yale (Graduate) School of Nursing and from 
other departments of the Medical Center and the University. The research unit 
will be headed by a director with a joint appointment at the Community 
Health Care Center Plan and the Department of Epidemiology and Public 
Health. The staffing pattern provides for research assistants, systems analysts, 
programmers, secretarial and clerical assistants, field personnel, etc. Physician, 
nursing and administrative consultants will be used whenever indicated. 
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Baseline studies have already been undertaken on the prevalence and 
characteristics of health insurance among the general population in the New 
Haven area; the cost of such insurance; the utilization of health services by 
illustrative samples of families and the costs; and the attitudes of families and 
various population groups toward medical care received and in regard to 
participation in a prepaid group practice program. These studies are planned on 
both pre- and post-enrollment bases. 

The record system has had and will continue to have considerable 
attention directed toward the feasibility and extent of automation. It will serve 
to record and integrate whatever data is needed: to facilitate the delivery of 
medical care to individuals served by the Plan; to operate the prepayment 
aspects of the Plan; to perform research studies which involve evaluation of the 
services provided by the Plan; and to make comparisons with data derived from 
other population groups not served by the Plan. The records will also be 
designed for use in the collection of demographic and medical care data for 
research and evaluation purposes. 

Evaluation of the effectiveness jf the medical records system will be 
performed through studies on the efficiency of the provision of medical care 
by the group practice teams; through the analysis of individual medical records 
for utilization review and case studies; and through the efficacy of the record 
system itself. 

This system is being developed under contract with the National Center 
for Health Services Research and Development, with provision for inter- 
locking, if feasible, with the systems of the Yale-New Haven Hospital, other 
local area medical care plans and C.UP.I.S.S. 

Training: For purposes of medical education and training, one of the 
fundamental objectives of this Plan has been to promote the linkage of a 
teaching medical center to a multispecialty group practice organized to provide 
continuing comprehensive medical care. Thus, a residency or fellowship 
program for the training of primary physicians is a logical extension of the 
Community Health Care Center Plan through its affiliation with the Yale-New 
Haven Medical Center. 

Not until most physicians-in-training have reached the advanced resi- 
dency or fellowship level are they ready to assume full responsibility for 
continuity and coordination of the care of patients. For those interested in 
training for primary care, we propose that the last one or two years of 
residency or fellowship training-in internal medicine and pediatrics-be 
divided equally between the Plan and the Medical Center. Patients will remain 
under the care of their own primary physicians after the residents have 
completed their tours of training, thus preserving elements of long-term 
continuity, of care for the enrollees. . In addition to offering expanded 
opportunities for the training of primary physicians, the Plan would provide a 
new resource for a portion of the Medical Center training programs in surgery, 
obstetrics and gynecology, radiology and psychiatry and possibly some of the 
subspecialties. 
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Undergraduate medical students and nurse-clinicians-in-training will have 
an important, although limited, involvement in the ambulatory activities of the 
Plan. Because students are slow to complete both routine and diagnostic 
workups, they would unduly prolong the amount of time spent by patients and 
families away from work, home or other activities. Nevertheless, both 
nurse-clinicians-in-training and medical students will benefit from the availabil- 
ity of the Plan in several ways: attendance at lectures and seminars on the 
provision of medical care, with the group practice, plan as a working model; 
observation of personalized patient care in a functioning ambuiatory group 
practice facility affiliated with a medical center; opportunity to examine 
member patients of the group practice plan when they are in the hospital and 
to observe their inpatient care; participation in the delivery of patient care at 
various levels— as assistants to staff physicians or nurse clinicians or as members 
of the team of supporting personnel; and investigation of clinical and 
epidemiological aspects of medical care for research projects at the under- 
graduate level. 

In addition to clinical training, an important function of the staff of the 
Plan will be to train public health residents or fellows from the Department of 
Epidemiology and Public Health or other schools of public health in the 
administration and delivery of medical care and in health education. Both 
administrative and clinical personnel should be available to instruct these 
trainees. 

The responsibility for funding and administering all Community Health 
Care Center Plan training programs should be under the auspices of the Plan in 
cooperation with the Medical Center. Income accruing from services by 
trainees will offset some portion of the training cost, but will need 
supplementation from other resources. 

An affiliation agreement with the Medical Center provides for assistance 
and recruitment of an adequate staff of physicians and other personnel 
qualified for professional, educational and research functions. It also provides 
for periodic review of the health services provided and the educational and 
training activities conducted by the Plan. General guidelines for the training 
programs are laid down by specialty medical boards and other medical 
institutional accreditation committees. 

OUTCOME MEASURES 

Effects of the Plan’s operation on the health status of its enrollees will be 
evaluated from data on the incidence, prevalence and prognosis of disease and 
by measurement of functional capacity, morbidity and mortality among the 
enrollee population— obviously after the Plan has grown to sufficient size and 
has had a long enough period of operation. We intend to make e*'ery effort to 
obtain similar data from other populations although measurements of the 
prognosis of disease will be much more readily accomplished within the Plan 
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than on comparable groups outside our enrollment. Additional study areas may 
be set up to explore a variety of problems and programs. 

Special approaches toward the measurement of quality of care and 
outcome will be through internal and external audit. We hope to compare staff 
performance within the Community Health Care Center Plan on the basis of 
standards now being developed to study the performance of some clinical 
faculty members at Yale University who are in community practice, and by 
comparison of performances within the Community Health Care Center Plan 
with peer standards of good medical care. These studies will be supplemented 
by an external audit of clinical care carried out by a joint review committee 
from the Medical Center and the Health Center staff. We will be aware, of 
course, that both internal and external medical audit influence outcome and 
therefore become variables affecting performance in comparison with a 
standard or an experience not subject to audit. 

One of the most important measurements of effectiveness of the New 
Haven experience will be the overall success of the Plan: its acceptance by the 
community at large, its productivity in manpower training and its ability to 
generate meaningful research data and evaluations. These measurements will be 
effected, in large part, through historical records. This can be done in a 
scientific fashion through observation of the experiences of the Plan-both by 
people directly involved in the Plan and by others qualified to observe and 
evaluate its operation. 

COST 

Computation of total costs must include not just start-up and operating 
expenditures, but also the costs required for the attraction of personnel, the 
pre-start-up costs and the costs of feasibility studies. In many instances, these 
are donated-in full or in part-through allocation of time and research effort 
by interested institutions and universities or by voluntary contributions of time 
from concerned citizens. At any rate, there will probably never be a full 
accounting of much of the expense in time, money and heartache of the many 
plans that never got started— not to mention existing plans, those which are 
near or in an early operational phase, and those which are not yet on the 
drawing boards. 

The New Haven Plan will make every effort to compute the total cost of 
actual start-up, and unit costs for services, wages, overhead, facilities and 
donated time for each of the operational areas of service, training and research. 
This accounting is being designed by ourselves with consultation from an 
accounting firm with special expertise in this field. Special studies should reveal 
the additional medical expenses incurred by Plan subscribers outside of the 
Health Care Center so that we will know the per capita cost in toto-not just 
the cost of services provided by or through the Plan. Also, special studies 
among nonenrollees should give us comparative figures for evaluations of what 
we are and are not achieving in cost controls. 




EAST BALTIMORE MEDICAL PROGRAM 



Malcolm Peterson, M.D. 

The establishment of a health care program in the East Baltimore 
community was conceived as a necessary component of the activities of the 
Office of Health Care Programs even before that administrative unit of The 
Johns Hopkins Medical Institutions came into being in February, 1969. The 
objectives of the Office of Health Care Programs in establishing the East 
Baltimore Medical Program have been to develop in the community surround- 
ing The Johns Hopkins Hospital a service program which would: 

• provide a higher quality of continuous, fajjiily-centered, comprehen- 
sive health services to a defined population. 

• be responsive to the perceptions and expectations regarding health by 
the population to be served. 

• provide a “community medicine” setting in which the research and 
educational objectives of The Johns Hopkins Medical Institutions 
would be sustained. 

• permit experimentation with staffing, financing, and organization of 
the delivery of primary care. 

• develop an alternative avenue for the delivery of primary care to the 
neighborhood residents who have 'been using, in increasingly large 
numbers, the Emergency Room and Outpatient Department of The 
Johns Hopkins Hospital as the number of local practitioners and 
hospitals have diminished, leaving this institution and its staff virtually 
the only provider of care to many thousands of nearby people. 

• prevent an increase in the institution’s fiscal deficit by rendering 
primary care and perhaps show how the monetary burden might even 
be reduced. 

• provide tangible evidence of genuine concern about the health 
problems of those people who have come to rely on The Johns 
Hopkins Medical Institutions for provision of services. 

• serve as a model of health care delivery which might be transplanted to 
other practice settings, locally or elsewhere, or to other teaching 
hospitals. 



ESTABLISHMENT OF THE PROGRAM 

Following preliminary discussions within the Institution, members of the 
Office of Health Care Programs met with a broadly representative sample of 
the East Baltimore community at a retreat held in Manresa House in Annapolis, 
Maryland. Emerging from this lively discussion was the awareness that the 
residents of the community harbored deep-seated resentment and distrust of 
The Johns Hopkins Hospital, but they felt that the deficiency of health services 
plus the expressed commitment of the institution demanded the development 
of a health care program which would be sponsored jointly by the community 
and The Johns Hopkins Medical Institutions. Considerable discussion about the 
geographic base for such a program led to the decision to embark on a program 
which would serve people in East Baltimore in an area demarcated by 
preexisting Model Cities boundaries. 

As the planning efforts developed, the problems of financing as well as 
the administrative and programmatic difficulties became increasingly apparent. 
The consequent temporal constraints on the actual initiation of delivery of care 
to this entire population led to our recognition of the necessity for a more 
realistic and tangible starting point. 

Accordingly, a population which was readily identifiable and already 
defined was selected by turning to city housing projects for low income 
families which were in or contiguous to the Model Cities areas already selected. 
A review of the records in the Office of Housing and Community Development 
permitted us to anticipate overall eligibility for various programs of payment 
for medical services as well as to quantify demographic determinants of service 
demands. 

Assembling the pertinent information on the 8,200 persons living in the 
four housing projects showed the same data which are depressingly common to 
most inner-city public housing groups; namely a median age of 14 years, 2/3 of 
families headed by women, median income of $2,700, median family size of 
3.8 persons. Extrapolation from these facts enabled us to recognize that only 
75 percent of the population would be eligible for the State Medicaid Program 
together with the Medicare and Children and Youth Program beneficiaries. The 
problem of securing funds for payment for services to the remaining 25 percent 
led us to confer ivith private insurance carriers as well as federal granting 
agencies. Initial discussions with Blue Cross and other private insurance carriers 
confirmed our expectation that funds for health services would not be 
forthcoming from such sources. 

In a series of meetings attended by representatives of the Office of 
Economic Opportunity, the Community Health Services (314e), Children’s 
Bureau, and representatives of the regional office of HEW, we concluded that a 
request for these funds directed to the Federal Government might receive 
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favorable action with the total sum forthcoming from this or some other 
consortium of agencies. Accordingly, a proposal was submitted in October of 



1969. 



BUDGET 

Using utilization data from the National Health Survey and the State 
Medicaid experience, we estimated, for a population of 5,000 individuals 
enrolled from the defined population, the costs of services as they would have 
been paid by the Medicaid Program iii Maryland. These include inpatient, 
outpatient, and pharmaceutical expenses. Dental care and preventive services 
were felt to be additional needs of the program, but payment for these would 
have to be arranged from the other sources of funding. 

In addition to the preparation of the proposal to the Federal agencies, a 
prolonged series of negotiations with the State Department of Health evolved a 
contract for the prepayment of the services for those enrollees who would be 
eligible for care under the Title XIX legislation. This agreement which is to be 
signed by all parties in the immediate future states that the Hospital will 
receive that amount of money which is equivalent to the average charges for 
care at The Johns Hopkins Hospital at the customary level of reimbursement. 
This sum of money, approximately $280 per year per person, will be paid 
based on the dates of enrollment of the recipients and verification of their 
eligibility for such payment of services at six-month intervals. Furthermore, the 
contract provides that for the initial year of experience the State will guarantee 
payment for those expenses which are allocable to the Title XIX Program but 
which were in excess of the anticipated operating budget. Should there be a 
savings in the cost for care rendered to Medicaid recipients, the agreement is 
for a return of 50 percent of this excess to the State with utilization of the 
remaining 50 percent for payments to the physicians as incentive reimburse- 
ment. Should the limits of such incentive reimbursement be reached, the 
remaining funds will be used in improvement of the services. 

The extrapolation of similar anticipated expenses for services to the 
remaining 25 percent of. the target population enabled us to establish the 
working budget to be supported through federal grants. 



ADMINISTRATION 

Throughout the evolution of these budgetary programs and consideration 
of the services to be delivered a significant dialogue with members of the 
community has been maintained. After some delays and travail resulting from 
our inability to discern who “represents the community,” a committed and 
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very strong-minded group of individuals emerged. Incorporated as the East 
Baltimore Community Corporation, these black people presented an unyielding 
but appropriate demand for a policy-making role in the evolution and 
operation of the East Baltimore Medical Program. Every word of each draft of 
the proposal submitted to the federal agencies and the contracts prepared with 
the State Department of Health was read aloud at meetings of the East 
Baltimore Community Corporation and attended by staff of the Office of 
Health Care Programs. 

The By-Laws of the East Baltimore Community Corporation were 
written by its members with a clear understanding on everyone’s part that this 
group would be constituted to establish more community service programs 
than just a medical program in East Baltimore. Nevertheless, this activity was 
the immediate reason for the existence of the East Baltimore Community 
Corporation and has occupied the complete agenda for each of the many 
meetings held thus far. 

Once the status of the East Baltimore Community Corporation had been 
established, a Joint Working Agreement with The Johns Hopkins Hospital was 
developed. Many of the considerations in the preparation of this Agreement 
and the By-Laws for the East Baltimore Community Corporation stemmed 
from guidelines of the varied agencies with which the program was dealing. 
Conflicts with the guidelines themselves or the intentions of these guidelines, 
the preferences of the East Baltimore Community Corporation, and profes- 
sional judgements by the representatives of The Johns Hopkins Hospital 
introduced many hours of controversy and dissatisfaction into the nego- 
tiations. Evolving from the negotiations, however, has been a clear agreement 
that the operating budgets will be prepared conjointly by the Administrator of 
the Program (who is responsible to the East Baltimore Community Corpora- 
tion) and The Johns Hopkins Hospital (represented by the Office of Health 
Care Programs). The Administrator will have responsibility for the day-to-day 
operation of the East Baltimore Medical Program, and he will follow policies 
which are established by the East Baltimore Community Corporation in 
agreement wi f h the Office of Health Care Programs. 



SERVICES 

The health care rendered to enroliees in the prepaid East Baltimore 
Medical Program will be rendered from a facility which is located in the 
neighborhood of the housing projects to be served. Enrollment will be by 
families and the enrollment process will include assessments of the present 
health status of the individuals in that household as well as some assessment of 
their previous utilization of sources of health care. 
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The health services are to be rendered through a group practice staffed 
primarily by internists and pediatricians working as part of teams which 
include outreach workers (family health workers), health care managers 
(primary care specialists and associates), and mental health technicians. The 
concept of the team which is coordinated by the primary care specialist is to be 
one of the main areas of evaluation in this program. 

The patients will identify their interface with the health care program in 
the form of the assigned family health worker. In turn, this family health 
worker will communicate directly with the primary care specialist to transmit 
information about the health status within the household. The primary care 
specialist on seeing patients assigned to him will delegate prevention, diagnosis, 
treatment, or other management of these health problems to the appropriate 
members of the team. The mental health technician, pediatricians, internists, 
obstetricians or nurse midwives, and social workers employed in the program 
will relate to the primary care specialists; team meetings will be organized as 
necessary to address the health problems of the household as a group. Any 
decision emanating from this meeting will be transmitted to the family or 
individuals involved by way of the family health worker who will be a full 
participant in the team meeting. Consultation services through the specialty 
clinics of the Outpatient Department will be rendered as necessary. Basic 
laboratory services and limited X-ray services will be provided at the health 
center. Hospitalization at The Johns Hopkins Hospital with continued care by 
appropriate physicians from the East Baltimore Medical Program will be 
provided as necessary. Medications prescribed by the staff of the East 
Baltimore Medical Program will be dispensed from a branch pharmacy located 
at the Medical Center. 

The format of the medical records will be the problem-oriented chart 
developed by Weed. A tabulation of the utilization of services will be 
maintained, relying on the format of the information system which is currently 
being developed for use in the Columbia Program. 



LESSONS TO DATE 



Problems encountered in the development of the East Baltimore Medical 
Program are those reported by others who have mounted similar endeavors 
elsewhere. We have encountered difficulties with funding, unfavorable re- 
sponses of local professionals, and disagreements with the community 
representatives. Thus far, none of these has proved insurmountable, and each 
of the encounters has resulted in consolidations, realignments, and amplifica- 
tions of the program to its benefit. 
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Aside from the difficulties of dealing with disparate sources of funding 
which result in incomplete coverage of the target population or limit the 
services to be rendered, the inadequacies of the payment for developmental 
and start-up costs of such a program have been painfully evident. We have been 
fortunate in working with some support through a planning contract for the 
East Baltimore Medical Program, but this has been only a fraction of the cost 
of development and can be applied only to the creation of the program on 
paper. The actual start-up cost of the program itself (losses during overstaffing, 
costs for construction of the facility, expenses involved with the recruitment 
process itself, the cost for enrolling patients, and the expenses of developing a 
community board of directors) are not supported through existing programs 
and are not recoverable from the service monies. 

The disapproval of the East Baltimore Medical Program by some of the 
local professionals who perceive a threat to their economic livelihood has been 
mitigated through extensive discussions with them about the nature of the 
program, its limited size, and their involvement. 

The mutual commitment to community participation led to the 
establishment of an agreed-upon program and a satisfactory working relation- 
ship between the East Baltimore Community Corporation and The Johns 
Hopkins Medical Institutions. This agreement and the growing mutual 
confidence have faltered when representatives of various agencies (each with 
different guidelines) have intervened to “clarify” the developing relationship 
between the community and the institution. 

EXPECTATIONS FOR THE EAST BALTIMORE 
MEDICAL PROGRAM 

In addition to achieving the objectives which were outlined above, we 
hope that the experience in the pilot phase of the East Baltimore Medical 
Program will provide a secure fiscal and medical operational base on which to 
expand to a five-fold larger scale of service. 

We hope that lessons learned through our experience in the East 
; Baltimore Medical Program might be useful to some of the public and private 

i third-party payors for health care. 

\ We anticipate establishment of training programs which will be advanta- 

i geous to the residents of the community in which this program operates. Such 

, training has already been initiated in conjunction with a local high school, and 

one of the East Baltimore Community Corporation members has taken this 
task as her major effort. 

We hope that the students in the health sciences at The Johns Hopkins 
University will be afforded ample opportunity to see group practice in 
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operation, comprehensive health services being provided, and practitioners 
functioning in a primary care setting. The research opportunities are manifold. 
We anticipate that these will be realized by the Health Services Research and 
Development Center of the Office of Health Care Programs and will be 
available to medical students and other interested researchers in health care. 

The accomplishments to date make us optimistic about the possibility of 
developing a much clearer understanding of the mutual problems and 
aspirations of the medical institution and the community in which it is 
situated. To this end, physicians practicing in the immediate vicinity of The 
Johns Hopkins Hospital have been given staff appointments. Other dividends 
are forthcoming as we encounter more frequent expressions of trust and 
respect from a community which has harbored considerable hostility to the 
institution. 



CONCLUSION 

The East Baltimore Medical Program, a project which has dual 
sponsorship from the consumers and providers of health care, has been in the 
early stages of development for almost one year. Lessons which we have 
learned to date reflect many of the problems inherent in the health care system 
itself, but they reflect primarily the disorganization of the programs which pay 
for health services to the lower economic classes in the United States. 
“Community participation” is a catch phrase which denotes a highly desirable 
objective, but carries with it the necessity for prolonged and involved 
negotiations. The rewarding aspects of a program such as this can be seen even 
before it becomes operational. The benefits which have been realized even 
before health care has been delivered prompt us to increase our effort to 
I achieve operation of a program which will permit teaching, accumulation of 
' experience, and research, and which promises greater community satisfaction 
with improved quality of health care. 

V. , 
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THE HARVARD COMMUNITY HEALTH PLAN: 

THE ROLE OF AN AUTOMATED MEDICAL RECORD SYSTEM 

IN EVALUATION 

Jerome Grossman, M.D. 

The specific aims of the Harvard Community Health Plan (HCHP) are to: 

• create an effective and economic means of organizing and delivering 
health care. 

• provide students, faculty, and staffs of affiliated medical institutions 
with an opportunity for studying and teaching comprehensive health 
care. 

• conduct social and clinical research concerning health care needs and 
effectiveness of health care. 



MEMBERSHIP 

Eighty percent of the target population will join through employer 
groups which offer the Harvard Community Health Plan through a dual choice 
mechanism sponsored by Blue Cross and the private insurance companies. The 
remaining twenty percent of the target population will be drawn from lower 
income neighborhoods enrolled through Medicaid, Medicare^ and grants. At 
present, there is no individual enrollment. There is a 15 to 30 day waiting 
period before coverage is effective. 



MEDICAL RECORD SYSTEM 

Before discussing specific plans for utilization and statistical reports, it is 
important to understand the medical record keeping system employed at the 
Harvard Community Health Plan. Because the plan was formulated as one 
deeply committed to research and teaching, it has worked in conjunction with 
the Laboratory of Computer Science at the Massachusetts General Hospital in 
designing and implementing a computer based medical record system. The 
system has been designed to serve primarily as a medical recordrin-contrast to 
a system designed for management-either administrative or fiscal-or for 
research. However, the system was also designed to h^ve maximum flexibility 
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and a structure such that data for utilization, administration and research 
would be readily obtainable. 1 



DATA COLLECTION 

Patient data, in coded or pre-structured form, are derived from three 
sources: 1) the enrollment card; 2) the health questionnaire sent out to all 
enrollees over the age of 18 (a questionnaire for children is being prepared); 
and 3) the patient encounter forms filled out for each visit. Data available from 
the HCHP Information System are presented in the following tables. 
Additional patient data from hospitals, extended care facilities, and out-of-area 
facilities are also collected on pre-structured forms. 



Table I. Enrollment Card 



A. Name 

B. Address (coded by ZIP code) 

C. Date of Birth 

D. Sex 

E. Marital Status 

F. Physician Group 

G. Effective Date of HCHP Membership 

H. Employer Group 

I. Source of Payment (Carrier and/or Mcdicare/Medicaid) 



Table II, Health Questionnaire 



A. Past Hospitalizations (Year and Reason) 

B. Medications 

C. Drug Allergies 

D. Family History of Disease 

E. Past History of Serious Illnesses 

F. Review of Systems 

G. Social History (Including Working Status, Occupation, and Educational Level) 



Table HI. Encounter Forms 



A. Administrative Data 

1 . Date 

2. Provider (Coded by Individual, Specialty, and Physician Group) 

3. Time, Site, Type of Visit 

B. Medical Data 

1. Diagnoses or Problem 

a. Activity 

b. Certainty 

c. Severity 

d. Progression 

2. Diagnostic Studies (Including Results) 

3. Medications 



At the time of enrollment each prospective member is assigned, an 
individual medical record number and the demographic data entered into a 
computer system. This establishes an individual patient file which can be 
indexed alphabetically, by employer group, or by any of the parameters 
included on the enrollment card. Samples for specific studies can be derived by 
terminal digit or by other randomization processes. Changes in basic 
demographic data or family group are entered into the computer system and 
new demographic output produced. Enrollees who discontinue the plan are so 
flagged in the computer record, but at present their records are maintained 
within the system. 

We have been in continual discussion with Blue Cross and with the 
employer groups to work out a comparison of our enrollees to the Blue Cross 
or privately insured population as a whole. At present, however, such data are 
not available for comparison. 



INFORMATION 

The encounter form deals primarily with coded information about the 
patient’s problems. The diagnosis codes were specifically compiled to be 
appropriate for ambulatory care. In addition to specific and established 
diagnoses, they include other types of problems, such as symptoms for which 
an etiology has not yet been established. For each problem, its certainty, 
severity, and progression are also recorded. 

As a step towards providing comparability of utilization data, this 
problem code has been developed in conjunction with the Bunker Hill Health 



Center. The diagnosis and problem codes have also been cross-correlated with 
ICDA codes so that comparison with other groups using these codes might be 
possible. Medications and therapeutic measures are pre-coded and are also 
included on this form. 

A second form filled out at the time of a patient encounter is a master 
schedule of activities, which is used to order diagnostic studies, consultations, 
return visits, and referrals, inside or outside of the center. There are also 
structured forms, by specialty, for the recording of physical examinations and 
pre-natal and post-natal data. In addition to filling out the above forms, the 
physician is given the option of dictating a note for insertion in the patient’s 
record. 

The two forms and subsequent test results are forwarded to the medical 
record room for entry into the computer system. With these data the computer 
then prints out: l)an encounter report, providing all the information 
pertaining to this encounter; 2) an updated summary of all diagnostic studios 
done to date; 3) an updated summary of the patient’s medication history; 4) a 
patient status report. The status report is a dynamic summary of the patient’s 
medical condition. Copies of the report are available in the office of the 
patient’s physician and at the central operator’s desk for use in giving 
emergency care. 

These four reports are inserted into the patient’s medical record. The 
transcription of the dictated note is attached to the bottom of the encounter 
report. The record is then placed in the doctor’s box in the record room for 
verification and signature. Once the reports are signed, the record is returned to 
the file, and the original coded sheets are discarded. 

EVALUATION 

This system provides the physician with a standardized, legible patient 
record and a current summary of a patient’s condition. But in addition to 
serving as the medical record, the data base is used to produce utilization 
statistics which are the core of the present evaluation of the plan. 

As yet we have no set schedule for tabulation of our data. In the first 
year we have produced monthly tables concerning: I) the rate of growth of the 
enrolled population; 2) the characteristics of that population; 3) the number of 
visits and number of people seen by service; and 4) the frequency of laboratory 
tests including X-ray and EKG. 

More detailed tables, such as number of enrollees by employer group and 
age, and medical usage by individual health professional, have been tabulated 
experimentally. Special purpose tables, such as a list of all enrollees who meet 
the State requirements for flu vaccination, have been produced on a one-time 
basis only. 
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ATTITUDES AND BEHAVIOR 



The record system at HCHP does not provide a direct measure of 
attitudes of either the staff or the patients. However, it does provide 
information concerning their behavior, in that it keeps: l)a record of broken 
appointments, by type of service and patient characteristic; 2) a record of the 
number of enrollees who have never used any services; and 3) a positive 
feedback on the staffs utilization of resources and facilities in terms of number 
of visits, number of persons seen, and number and type of tests and 
medications ordered. 



HCHP’s contract with the insurance companies calls for annual reporting 
of utilization statistics for employer groups. These data are used in adjusting 
the basic capitation rate. If utilization by a particular employer group is more 
than 10 percent above the estimated utilization, the insurers will increase the 
capitation rate paid to the plan. In the same manner, if utilization is more than 
10 percent less than estimated, the capitation rate is adjusted to a lower figure. 
If desired, costs can be broken down by: 1) patient characteristics (e.g. all 
patients older than 65); 2) diagnostic classifications (e.g. all patients with 
diabetes); or 3) functional categories (e.g. the number of physician services by 
individual practitioner or specialty group, or the numbers and types of 
diagnostic studies performed). 

The plan also provides services for a small number of fee-for-service 
patients. The costs for this group are kept separately. 



In this first year we have concentrated our efforts on creating a smooth 
system for the delivery of patient care. A most important aspect of this 
operation is to guarantee. the quality of the information we collect; while the 
computer can accurately store and retrieve data, human beings determine their 
quality. Therefore much effort has been put into examining the procedures for 
collecting the data and into verifying the data which has been entered into the 
computer. 



1. For further information about the computer-based record system, write the Laboratory 
of Computer Science, Massachusetts General Hospital, Fruit Street, Boston, Mass. 
021 14, attention J.H. Grossman, M.D. 
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THE MEDICAL CARE GROUP OF WASHINGTON UNIVERSITY: 
DEVELOPMENT, GOALS AND EVALUATION 

Gerald T. Perkoff, M.D. 

Lawrence Kahn, M.D. 

The Medical Care Group of Washington University (MCG) is an 
experiment in medical practice developed by the Division of Health Care 
Research of Washington University and supported by grants from the 
Metropolitan Life Insurance Company, the W. K. Kellogg Foundation, and the 
National Fund for Medical Education. Several defined goals were implicit in 
the development of MCG and others became apparent during the detailed 
planning of the medical care program. The plan was developed to compare in 
study and control patients the effect of an experimental ambulatory health 
insurance on hospital use and health status. We also hoped to develop a system 
of family-oriented group practice that would use but would not duplicate the 
special skills available in the medical center. It soon became evident that it was 
important to have sound data about utilization and costs of medical services. 
Such data should identify any savings of hospital expense and show what 
portion of such savings would have to accrue to a medical group before that 
group would be on sound fiscal grounds. We also wanted to learn the size such 
a group would have to be in order to make a significant contribution to the 
medical care capabilities of the community, and to gain information upon 
which a plan for improved care of indigent patients could be based. Other goals 
were to begin a practice which might become a base for teaching comprehen- 
sive health care, to provide a nucleus around which faculty interested in health 
care research could be assembled, and to stimulate health care research in 
Washington University School of Medicine. 



THE MEDICAL PLAN 

The Medical Care Group is a family care plan, staffed by specialists in 
medicine, pediatrics and obstetrics-gynecology. It incorporates screening and 
preventive medical care, diagnostic and therapeutic medicine and continuing 
medical care in one unit, with the aim that both general and consultative 
specialty services will be provided in a single location. Since the research 
program to be operated in conjunction with MCG demanded that the medical 
care delivered be monitored carefully, that accurate records be kept, that a 
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satisfactory control group be defined, and that! acceptance by the medical 
faculty and community of practicing physicians be as smooth as possible, it 
seemed essential to keep the practice small. Therefore, MCG will have a 
maximum of 500 study families and a comparable number of control families. 

The study families begin care in MCG by appointment for a complete 
history and physical examination and screening evaluation. If they must be 
seen for an illness before their complete evaluation, a later appointment for 
their complete check-up is made at the time of the initial visit. The internist 
who first sees an adult patient remains that patient’s personal physician; the 
same applies in the case of the pediatrician who first cares for a child. Control 
families, selected by lottery from the initial enrollment at the time study 
families are chosen, are given the same complete evaluation as are the study 
families, and all the information obtained is made available to their private 
physician for use as he sees fit. Further details of control selection and 
evaluation will be given later. 

All regular office medical services are provided in MCG offices, and 
physicians are available for patient care 24 hours a day, seven days a week. All 
hospitalization needed is carried out in the Barnes-Washk.gton University 
Hospital group, and MCG physicians hold both hospital staff and Washington 
University faculty appointments. The patient’s personal MCG physician 
remains responsible for his care when he is admitted to the hospital. 

When a consultation in any of the specialties not represented in MCG is 
needed, it is made by the appropriate specialist from the full-time medical 
school faculty. The consulting physician may see the patient in the MCG 
offices or in his own office in the medical center. Needed surgery is performed 
by surgeons of the Department of Surgery senior staff, with the MCG physician 
maintaining direct patient contact during the surgical admission. Follow-up 
care is provided by the MCG physician in the MCG offices. 



THE INSURANCE PLAN 

The insurance plan was developed as an integral part of the medical care 
plan with a number of considerations in mind. We knew that there was ample 
experience with preexisting prepaid group practice plans and that new large 
. experimental programs were being set up elsewhere. 1 For this reason a full 
prepayment capitation plan did not seem desirable for MCG. Also, particularly 
in view of the small size planned for MCG, we wished to study a payment 
mechanism which was intermediate between fee-for-service practice and full 
prepayment, and which might be more readily adaptable to present methods of 
medical care organization and practice. 



48 



53 



The plan developed in response to these needs uses the principle of 
prepayment by the insurance mechanism, with the funds received by MCG for 
services rendered, negotiated between the insurance company and MCG on the 
basis of current medical fees used in the medical center. Actuarial estimates of 
utilization provided the basis for these negotiations, and appropriate adjust- 
ments are planned at regular intervals. The current payment is $12.75 per 
month per covered family for the services listed in the following paragraph. 
MCG and control patients retain the same basic hospital plan they had before 
enrollment and continue to pay for that plan by whatever arrangements were 
available to them before MCG was begun. The hospital plan costs are separate 
from the MCG costs and the only difference between the study and control 
groups is the availability of the MCG plan to the study patients. 

The physicians who staff MCG are salaried, but MCG pays for laboratory 
procedures, X-ray services, consultations, surgery and other professional 
services provided to MCG patients on fee-for-service full charge basis. The 
services provided in the plan include: office visits in MCG offices; night, 
weekend, emergency and if needed, home calls by MCG physicians; hospital 
visits by MCG physicians; preventive check-ups; immunization and allergy 
treatments; ambulatory X-ray and laboratory services; all specialty consulta- 
tions; pre- and post-natal and pre- and post-surgical care; out-patient 
psychiatric treatment when the patient is referred by MCG physician; 
rehabilitation evaluation and short term treatment. 

Dental care and refractions are excluded as is cosmetic surgery, unless the 
need for such surgery arose from an accident or is necessary for the psychiatric 
rehabilitation of a child. 



SELECTION OF PATIENTS AND CONTROLS: ENROLLMENT 

Employees from three companies were offered the MCG plan. Each of 
the companies provided a basic health and hospital plan for their employees, 
some portion of which was carried by the Metropolitan Life Insurance 
. Company. The first company was Metropolitan life Insurance Company itself, 
which offered the plan to its local employees in St. Louis; the second was 
General Motors Corporation, which offered the plan to its St. Louis hourly 
workers as an additional option for health care under their United Auto 
Workers contract; and the third was the Monsanto Chemical Company. 
Enrollment procedures varied among the companies. In the case of the 
Metropolitan Life Insurance Company, the campaign was conducted through 
the district offices by the district managers, who were given detailed 
information about the MCG plan prior to the enrollment. Each district 
supervisor or his designee visited the MCG offices as part of the learning 
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process; sessions were held by Metropolitan group health personnel at the local 
Metropolitan offices as well. When enrollment was begun, the employees 
offered the plan were told that only 100 families would be selected. Over 200 
indicated a desire to join the plan and names of 100 study families were drawn 
by lot to select the first members. The remaining employees who enrolled in 
the plan were asked to serve as the control group for the Metropolitan 
employees who now were study patients. In the case of General Motors, the 
company and the local union leaders offered the plan to their individual 
chapter members without a marketing or sales campaign, by posting a bulletin 
board notice. Only SO families enrolled, and therefore no control group is yet 
available for this group of patients. In the case of Monsanto Chemical 
Company, the personnel managers of the various district offices visited MCG, 
were given the information about MCG services, and they then conducted the 
enrollment in their own divisions. In this instance almost 600 people enrolled 
for 150 open places. As in the case of Metropolitan employees, a control group 
will be developed from the remaining enrolled persons not chosen to be MCG 
members in the lottery. 

Only general comments can be made about the enrollment. In each case, 
interest in MCG appeared to be in direct proportion to the level of 
understanding of the plan gained at the informational meetings. For ethical 
reasons, neither the MCG administrative nor medical personnel were involved 
in. the enrollment procedures. 

Following enrollment, all family members become active MCG members 
at a subsequent prearranged starting date. The enrollment card itself serves to 
initiate the MCG record. Family membership and other demographic and 
identifying data are transferred directly from copies of enrollment cards to the 
patient’s plastic ID card and to his medical record. Enrollment is kept 
up-to-date by monthly notice from the insurance company. If enrollees 
discontinue the plan or transfer to another area, their health insurance reverts 
from the MCG plan to the basic plan as it was before MCG was initiated. MCG 
remains responsible for the care of members who discontinue enrollment, until 
48 hours after such time as notice is received at MCG of the fact that the 
employee has discontinued coverage. Several of- the initial enrollees discon- 
tinued the plan shortly after enrollment. In most of such instances the enrollee 
had not realized he would have to give up a relationship to his own physician at 
the time he became an MCG member. This was particularly likely to result in 
discontinuance if the patient was undergoing psychiatric treatment and had a 
productive relationship with his psychiatrist. Only in one or two instances was 
the patient dissatisfied with MCG as he found it. 
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EVALUATION OF THE MEDICAL CARE PROCESS 

The evaluation of medical care utilization is a major aim of the MCG 
program, and a specific system has been set up to allow retrieval of the desired 
information. For each patient encounter, the physician, or in the case of 
non-physician services, the nurse, fills out an events form indicating the services 
and procedures given the patient and these then are transcribed to a 
standardized record form (a code sheet for medical services and diagnoses), and 
classified according to a 28-part coding system (Table 1). Diagnoses are coded 
according to the International Classification of Diseases, procedures are coded 
according to the California Relative Value Scale, and the physicians' names are 
entered. For each service the standard fee also is entered. In the case of 
MCG-rendered services these are the charges MCG would have made for its 
services if it were a fee-for-service practice. In the case of X-ray, laboratory or 
consultative services the actual fees charged to MCG are entered. The recording 
forms are submitted monthly for computer storage and analysis, and desired 
information is generated at quarterly intervals. 

In contrast to MCG office visits, accurate recording of patient visits to 
consultants has presented certain problems. A simple MCG consultation form 
provides good access to information about the first contact with a consultant, 
and is returned promptly to MCG by the consultant. But information from 
subsequent visits to those consultants has not been readily available to MCG. 
Such data now are being collected weekly by a research assistant by telephone. 
At the present level of operation of MCG, this has been an effective technique, 
but this only will be so in a small operation; other possible solutions to this 
problem are under study at this time. Hospital utilization data from study and 
control groups are available from Metropolitan Life Insurance Company for 
patients who carry a Metropolitan hospital insurance plan. Where another 
carrier has the hospital plan, such data are obtained from hospital records. 

A checklist medical history has been adapted from one in use at the 
Ogden Clinic, Ogden, Utah, for submission to patients before they come to the 
MCG offices for screening examinations. A schedule for examining social and 
behavioral factors and attitudes of enrollees also has been developed but is not 
yet available in printed form. This questionnaire was reviewed by Drs. Odin 
Anderson and Merwyn Greenlick and then was modified, pretested and 
shortened. The schedule will be used both with study and control patients to 
provide data which can be the basis for future research. 



MEASURES OF OUTCOME 

The measurement of outcome is the most difficult and problematical 
aspect of the MCG program. Two approaches are being tried. For evaluation of 
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the medical care given in MCG itself, an audit system applicable to office 
records has been developed based on “acceptable local standards.”(2) This 
audit system has just been initiated, and it is too early to tell whether or not it 
will be effective or how much it will have to be modified. But it is hoped that 
regular estimation of performance against pre-set standards will provide a 
measure of the level of medical practice of the physicians in the group. It is not 
certain whether or not it will be possible to apply the same audit to control 
patients’ records. Some private physicians may be willing to have records 
audited, but many will not make their records available to us for study. 

A key aspect of the evaluation of MCG is the knowledge that comparable 
control families will be available for each enrolled group. The screening 
examinations will permit an estimate of the degree of comparability of study 
and control patients and will allow reasonable determination of health status in 
the two groups. Data from the screening examinations also are coded for 
storage and retrieval, using a sample form which indicates the presence or 
absence of abnormality. This should make it possible to interpret on factual 
grounds any similarities or differences found in hospital and medical care 
utilization. 



IDENTIFICATION OF COSTS 

The MCG accounting system is that used for research grants accounting 
in the medical school. Budget estimates are made in specific categories and 
code numbers are assigned for billing and record purposes according to 
projected needs in each category of service, such as consultations, laboratory 
and X-ray. Payments received from the insurance company for covered families 
are deposited in the MCG account, and payments and transfers are made either 
to other medical school accounts or to other providers of services or suppliers 
by voucher, each of which indicates the appropriate information for 
accounting purposes. Thus, when a patient is seen in consultation the amount 
of the consultation fee is transferred from the MCG account to the 
departmental fund of the consultant involved. Receipts and expenditures in 
each category will be available at regular intervals. 

The method for coding fees supplements and extends this system. As 
described earlier, when the patient is seen in the MCG office the record form 
shows not only the service rendered, the diagnoses made and the payments 
made by MCG for services rendered by others, but a fee also is entered for 
MCG services as though MCG were a standard fee-for-service practice. The 
actual costs of medical care will be compared to those which would have been 
obtained if practice were completely on a fee-for-service basis. With this data 
available it should be possible to calculate the volume of operation and level of 
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hospital savings which will be necessary for MCG to be solvent. In this regard it 
is important to note that MCG’s start-up costs are known and that all costs 
1 elating to the practice operation are kept separate from those of the research 
effort. Thus, office rental, phones, salaries, supplies and other incidental 
expenses related to the practice itself are and can be computed as operational 
costs. 



RESEARCH AND TEACHING 

Research is the basic responsibility of the Division of Health Care 
Research which is operating MCG and therefore is an integral part of MCG as 
already outlined. A broader research program in ambulatory care also is 
underway in the Division. Specific research projects in organization of rural 
health care, application of allied health personnel to medical care problems, 
sociologic and planning studies in community medicine and others are 
underway. The research personnel in the unit represent the disciplines of Social 
Work, Sociology, Economics, Communication and Medicine. Each member of 
the Division holds an appointment in a parent university or medical school 
department and an Associateship in the Division, and has teaching responsi- 
bilities in his basic department. The research unit, therefore, is closely related 
to the medical care plan, the university, the medical school, and the hospital. 

The Division also is responsible for teaching of the comprehensive and 
social aspects of medical care and medical care organization. We therefore plan 
to make MCG an integral part of our teaching program. Seminars and research 
electives in health care now are offered by the Division. An elective titled 
“Clinical Preceptorship in Medical Practice” has been added for the coming 
academic year, in which senior students will practice for six to twelve weeks in 
the private offices of the physicians who staff MCG. After the details and 
techniques of this elective are worked out, we plan to. offer the same 
opportunity in MCG with the same physicians. Other teaching efforts also are 
planned. 



FINAL COMMENTS 

We believe that it is important to develop programs like MCG within the 
medical center, both to test possible improved methods for delivery and 
financing of medical care and to provide within the center a basic program for 
teaching of medical care. We hope this kind of unit will be a valuable 
laboratory for the study of medical care, and a meeting ground for clinical 
practitioners and academic physicians in the area of comprehensive medical 
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care. Further, we hope the data we obtain will allow us to develop functional 
plans for ambulatory care of indigent patients within our medical center Our 
initial experiences make us optimistic that our stated goals can be achieved. 
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Table 1 

TYPES OF SERVICE CODES 



Surgery 1 

Medical m Hospital 

(non-psychiatric) 2 

Immunization 3 

Anesthesia 4 

Night and Weekend 

Emergency 5 

Diagnostic X-ray 6 

Radiation Therapy 7 . 

Diagnostic Labc:atory 8 

Consultation - Surgical 9 

Surgical Assistant 10 

Psychiatric - First 

5 Visits 11 

Psychiatric - 6th and 

Subsequent Visits 12 

Consultation - Medical 

(non-psychiatric) 13 

Medical in Office - 

First Visit 14 



Medical in Office - 

Subsequent Visits 15 

Medical in Office - 

Preventive 16 

Rehabilitation 17 

Medical - Screening 18 

X-Ray - Screening 19 

Laboratory - Screening 20 

EKG - Screening 21 

Gynecology - First 

Visit 22 

Gynecology - Subsequent 

Visits \ 23 

Obstetrical - First 

Visit 24 

Obstetrical - Subsequent 
• Visits 25 

Postpartum n 26 

EKG - Diagnostic 27 

Office Procedures 28 
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A TRIAL METHOD FOR COMBINED END RESULT 
AND PROCESS EVALUATION 
OF HEALTH SERVICE SYSTEMS ) 
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Diana Ten Houten, M.A. j 

Donald M. Du Bois, Dr. P.H. 
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With the establishment of. the California Center for Health Service 
Research at UCLA, campus-wide interest was stimulated in the development of 
experimental models of health service systems designed to realize the fullest 
potentialities of modern medical technology in promotingmealth. Three such 
models have been conceived and are presently being pressed toward implemen- 
tation. 

Model One: A Child Health Care Prepayment Plan for Children of 
Students at UCLA - This model was developed as a pilot program in direct 
service prepayment. Through the Department of Pediatrics and the Marion 
Davies Clinic for Children, nearly comprehensive health services for the 
children of students are being offered on a contractual prepayment basis. Care 
under this plan will commence effective April 6, 1970. It is expected that 
initial enrollment of approximately 400 children will grow to an enrollment of 
2,000 children in the fall of 1971. By that time it is anticipated that provisions 
will have been made for prepaid health care services for the spouses of married 
students as well, and that a comprehensive student-family health service 
potentiality on the UCLA campus will have been created. The evaluative 
techniques developed under this research proposal will be applied in the 
evaluation of this prepayment model. In addition, if it is necessary, the model 
may be used as a laboratory in the development of the techniques themselves. 

The Regents of the University of California took formal action in 
February, 1970, authorizing the President of the University to approve the 
proposed Child Health Care, Plan. In doing so, the Regents acknowledged the 
principle of contractual direct service prepayment relationships as one of the 
acceptable alternative paths for the financing and operation of teaching 
hospitals associated with the University of California and its schools of 
medicine. 

Model Two: Extension of the University Resources to Augment the 
Primary Health Care Resources of Certain Areas Deficient in Primary Care - 
UCLA Hospitals and Clinics and the School of Medicine have been involved 
with an increasing intensity in the planning of a program to augment the 
primary health care serviced available in the South Santa Monica, Ocean Park, 

55 



GO 



and Venice areas of Los Angeles County. These areas represent concentration 
of low income families and welfare recipients. The primary health care 
resources are notably deficient. In cooperation with the Los Angeles County 
Health Department and the Los Angeles County Department of Hospitals, 
efforts are presently underway to formalize the development of community 
health resources in the West Los Angeles area to meet the deficiencies in 
primary care in these specific areas. UCLA has agreed to provide medical staff, 
specialist backup, and hospital facilities support for the development of a 
comprehensive health care clinic by the Los Angeles County agencies 
concerned with health. In order to finance the development of such a model 
program, the concept of contractual prepayment for categorical aid recipients 
under Title XIX (MediCal) is being developed. It is expected that County 
Health Department services on a comprehensive care model will commence in 
the South Santa Monica area in May of 1970, Because of the potential use of 
prepayment as a mechanism for financing care in these new primary care 
involvements, this model also provides a laboratory in which the population at 
risk will be known in advance. The model, therefore, will also be capable of 
serving as a laboratory within which some or all of the research proposed may 
be carried out. 

Model Three: A University-Community Health Service Prepayment Plan 
A prepaid health care program is to be offered on a voluntary enrollment 
basis to university faculty and staff and to others in the university community 
who wish to identify themselves with the UCLA Center for Health Sciences as 
their principal source of health care should they become ill. Enrollment for this 
plan is to be conducted by Blue Cross and Blue Shield both on the UCLA 
campus and in the outside community. To serve enrolled members, a new 
family health care center will be developed as a separate health care entity. 

The initial enrollment from the UCLA faculty and staff potential of 
30,000 persons is expected to be approximately 4,000 persons. Blue Cross and 
Blue Shield marketing commitments are for the development of approximately 
700 persons per month additional membership during the first year and 1,000 
persons per month additional membership during each of the succeeding two 
years. Enrollment is expected to reach 36,000 persons in November of 1973. 

The Family Health Care Center will be staffed by 21 primary physicians 
(internists, pediatricians, and specialists in family practice) and the necessary 
supporting specialists and staff. At full operation, a complement of psychiatric 
resources, full-time surgical consultation, and obstetrical and gynecological 
consultation and services will be directly available at the Family Health Care 
Center. Other specialty service and inpatient care will be provided through the 
extensive resources of the Center for Health Sciences. The Family Health Care 
Plan has been conceived not only as a model program to create the ideal setting 
for the physician-patient relationship in an organized context, but it has also 
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been planned as a laboratory. The subdivision of the 21 primary physicians 
into three teams of seven physicians each; the fact that each serves its selected 
population on a continuing and responsible basis, that each team serves the 
entire family unit, and that members are free not only to select the team that 
will serve them, but to change teams in the event of dissatisfaction, creates a 
setting within which considerable controlled experimentation may be done 
with respect to the organization of health services. Changes in staffing patterns, 
hours of operation, methods of handling patients, applications of new 
technology, etc., may be tested under conditions using the other similar teams 
and their populations as controls. It is believed that as many as six 
simultaneous experiments relative tc the organization and delivery of health 
services may be carried out within this one setting simultaneously without 
confusion of results. In order to develop and use this laboratory in this way, 
however, it is imperative that baseline performance data be collected and that 
the criteria for outcome be developed. 



THE EVALUATION DESIGN 

In planning the evaluation research for these programs, there have been 
two basic goals: 

• an adequate assessment of the end results of the medical care program; 

• development of a general methodology for the measurement of 
process problems in such programs. 

If the model to be tested is valid, it should be possible to measure 
program effectiveness for the current program while testing the model. 

The Development of a Model : Evaluation of health service programs or 
systems has followed three forms. Donabedian 1 classifies these forms as 
appraisal of structure, assessment of process, and assessment of outcomes. The 
bulk of formal efforts at the evaluation of health service systems or 
components has been placed on appraisal of structure in the form of 
certification and accreditation processes. It has been noted that such appraisal 
of the presumed capability of a system to perform effectively does not directly 
measure its actual performance. 1 ,2,3,4,s 

Assessment of process, the direct or indirect observation of the 
transactions occurring within a health service, system or program, is a more 
proximal measure of the actual performance of a health service system. But as 
Donabedian has noted, process evaluation is dependent upon comparison of 
the observed performance with a model of “good” or another observation set. 
Invariably, however, that which is observed in process evaluation is determined 
by an assumed relationship betv/een “good process” and successful health 
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outcomes or end results. Health service process evaluation is presently an 
expanding field. The landmark developments by Peterson, 6 Lembcke, 7 and 
Morehead and Associates 8 ’ 9 ’ 10 ’ 1 1 have opened up the enormous potentialities 
of observation of the physician-patient interaction process and the possibility 
of drawing evaluative conclusions about the technical quality of these 
interactions. It may be noted, however, that process evaluation has almost 
exclusively been limited to the single dimension of technical quality in health 
service programs. In the last two decades, increasing attention has been given to 
the multiple dimensions of quality in medical care. In calling attention to such 
dimensions of quality as accessibility, acceptability, continuity, coordination, 
comprehensiveness, and efficiency, the Program Area Committee on Medical 
Care Administration of the American Public Health Association (a Guide to 
Medical Care Administration, Vol. I) has cited the importance of a broader 
concept of medical care process. 12 

Process evaluation as it has been developed thus far has tended to have 
two major problems: 

• It has focused upon medical care process within the professional 
sphere of activity to the exclusion of those * process problems or 
failures which occur outside the walls of the institution. This problem 
has resulted from: (1) the predominant use of the medical record as a 
source of information; and (2) a tendency to define the health service 
system only in terms of the professional components thereof., The 
analyses by Parsons (The Social System) 13 and Freidson (Patient’s 
Views of Medical Practice) 14 lead clearly to the' conclusion that any 
definition of the health service system must include the population at 
risk. Both sociologists clearly support the premise that family and 
friends serve as the first validation and intervention screen of the 
health service system. 

• The inability of the usual institution, representative of the dominant 
medical care system in the United States, to identify its “population at 
risk.” Diffuse relationship patterns, in which an individual is. not 
identified with any specific service component until such time as the 
need for that component arises, make it impossible to identify 
segments of the health service system external to the professional 
setting until such time as the need for care has been defined by the 
patient, and professional care is sought. The existence of longitudinal 
contractual relationships between the population to be served and the 
professional components of the health service system through direct 
service prepayment relationships introduces the opportunity to extend 
1 salth service process observations well beyond the realm of the 
in-house professional performance. 



The multidimensionaiity of process qualities leads to a further problem 
in health service process evaluation. Tire relative importance of process failures 
in accessibility or acceptability compared to process failures in technical 
quality can best be judged in terms of the degree to which those problems or 
failures tend to have an impact upon the health of the population to be served. 

Assessment of outcomes has followed four dominant paths, each 
representing only a single dimension or fragmentary approach to estimation of 
the efficiency of effectiveness of health service systems or programs. 

• A number of studies have focused on the outcomes of intermediate 
processes . 1 5 6 7,1 8 

• Specific programs have been evaluated in terms of costs related to 
effectiveness. In such programs the population at risk is defined as an 
entire community and changes in the community level of the disease 
at which this specific program is aimed are the outcome criteria. 

• The outstanding works of Densen, Shapiro and others L examining 
perinatal and maternal mortality , 1 9 > 20 ,2 1 ,22 ,23 ,24 general mortality 
within an indigent population, 2 M 6 and morbidity data 27 as out- 
comes, are an example of the third form of assessment of outcomes. In 
such studies the ability of programs to define accurately," and 
prospectively the population at risk has allowed t he observers to 

. determine comparatively the impact of the organization and its efforts 
upon certain aspects of the health problem. 

• The fourth dimension of outcomes assessment which has been 
frequently applied (though with questionable effectiveness) is cost 
assessment. Under the implicit assumption that health status outcomes 
are comparable, thd costs of two programs have frequently been used 
to make judgments about policy. The limitations of this reasoning 
need not be labored here. 

End results measurement has been slow to develop as a tool for 
evaluation of specific subsystems of the health service industry. Physicians have 
been reluctant to be judged on the basis of the outcome of any specific case or 
any set of cases. They recognize realistically that the best medical practice is 
predicated upon probabilities of outcome which are always uncertain at the 
medical practice level. In addition, outcome evaluation hinges on the ability to 
attribute good or bad outcome to the performance of a specific health service 
provider or provider system. While outcome evaluation has been applied in 
terms of the mortality and morbidity of a given community, and comparative 
judgments made between the performance of service systems in different 
communities, the absence of any community-level health service management 
and the usual inability to attribute responsibility to subsystems within the 
community have made it unlikely that outcome evaluation in terms of 



mortality and morbidity could be usefully brought to bear on the diffuse and 
informally organized dominant medical practice-community. Nevertheless, the 
growth, on the American scene^ol-orgamzed medical practice entities, the 
gradual appearance of^Tgarnza^ons which could be identified as health service 
“firms,” the growth of patterns of contractual direct service prepayment 
relationships between health service firms and their populations at risk and the 
pressing national need for discovery of organizational patterns yielding 
effective and efficient health service have now created a favorable environment 
within which to further explore assessment of outcomes. 

This paper suggests an approach which combines the measurement of 
effect on health status with process problems analysis. It is hoped that the 
combined . approach will yield not only a new set of measures of the 
comparative effect of different health service enterprises, but also a new set of 
management information. Quantitative and qualitative analysis of the ways in 
which the system has failed to live up to the expected ideal will provide 
management with a new kind of tool for decision making. Both Shapiro and 
Donabedian have noted the “need to include both process and end results in 
systems of continuous or repeated appraisal” of health service systems. In the 
proposed research, simultaneous efforts will be made to quantify the multiple 
attributes of health service process, and the end results of that process. 

The proposed research rests upon a view of the health service system as 
“an agency of social control.” It is seen as a formal control system dealing with 
the “deviant state” of illness in the same sense that the education system deals 
with the deviant state of ignorance or that the law and its agents deal with the 
deviant state of criminality. The theories expressed by Talcott Parsons (The 
Social System, etc.) give rise to this formulation and suggest two solutions for 
critical problems at this time. 

First, it provides a firm footing to establish an overriding social concern 
for the performance of the health service system. This creates the opportunity 
to support the replacement of market demand criteria with other criteria for 
allocation of scarce health resources . 28 The application of cost-effectiveness 
studies' at the system level then becomes more practical. It is a principal tenet 
of this paper that tne operational goal of the ideal health service system is 
control over the health impairment burden. 

Second, Parsons’ formulations call attention to the important function of 
“family and friends” as the first referral screen of the health service system. 
Freidson’s research demonstrates the utility of this concept and the implica- 
tions of the theory for the organization of health care. This research proposal is 
based upon a view of the health service system as four concentric rings 
representing from the center out, institutional care, specialist care, primary 
professional care ; and the arena of family and friends as a first source of 
validation of illness and referral to appropriate action. 
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The overwhelming tendency of past evaluation efforts has been to view 
the health service system as limited to the professional arena. Family and 
friends are external and are seen as potential patients and as influences, but 
seldom as parts of the system. When one; considers eacli ring of the system as a 
screening mechanism which should keep the service involvement at the lowest 
competent level, the importance of the knowledge and function of family and 
friends becomes clear. 

The estimating of health impairment for the study population is the first 
research task. It will be approached from several directions in order to achieve 
accuracy in the estimation of validity, reliability and sensitivity. Although one 
estimate of health impairment might be obtained from the examination of 
medical records this procedure would assume that almost all health impairment 
came to the attention of the medical care system which is probably not the 
case. Thus it is imperative to gather data directly from the population. A 
thorough assessment of health impairment of one case will require the 
combination of intensive interviewing, observation and review of the medical 
record. One task of this research effort will be to determine a format for 
obtaining such data which is within the grasp of an ongoing medical care 
program. This requires, first, the development of a feasible interview 
instrument and second, it requires an analysis of that interview data compared 
to medical record data in an effort to identify segments of data which can in 
the future be taken from the management information system. 

It is understood that the concept of health impairment itself holds 
several possible pitfalls. The impairment experienced by different individuals 
having the same medical problem may differ greatly as a product of that 
individual’s normal roles and functions. That -is, a young receptionist may 
suffer more objective as well as subjective impairment from a facial scar while 
waiting for plastic surgery than will a middle-aged male manual laborer. It is 
also true that impairment may be experienced differentially in different 
spheres of an individual’s life. Thus, it is important to assess impairment not 
only in the ability to function normally at work, but also to function in other 
social roles. These measurement problems may be handled through the 
effective combination of medical evaluation of impairment and individual 
evaluations of impairment made by respondents. 

The second critical concern of this research must be assessment of 
at t ributafcili ty of health impairment burden to the medical care system. Again, 
this will be accomplished through a combination of technical approaches. It 
will be necessary to thoroughly define and establish the criteria by which 
impairment is judged to have been avoidable, reducible, the result of the health 
service system, or appropriately attributable as an incentive for further 
concern. 



To a certain degree attributability may be inferred from the relationship 
between process problems and outcomes of the medical care system. The 
identification of process problems again requires the application of at least two 
techniques. Some process problems will be identifiable only from examination 
of the records while some process problems will only be apparent.. from the 
testimony of respondents. An additional set of process problems may be 
suggested by the existence of conflicting perceptions between respondents and 
records. 

Method of Procedure: The design of a study based on the foregoing 
objectives requires a multi-method approach. The methodology to be devel- 
oped in this research will combine three basic components: a longitudinal 
interview survey of a sample of program members and of samples from control 
populations; an examination of medical records supplemented by a subsample 
of physical examinations; and a laboratory study in which controlled 
experiments with alternative program styles can be performed. This component 
will require the application of observation, interviewing, ar.d other related 
research techniques. 

The Health Service Process Problems Study: The use of “Problems” as 
management guides and stimuli is ubiquitous. Problems are observed (or 
perceived) deviations from some “ideal” concept. Problems, where they can be 
noted and interpreted properly, aie our most useful guide for achieving or 
approaching the ideal. 'Hie use of “problems” in health care process as guides 
to approaching an “ideal” process is neither new nor startling, but the 
proposed attempt 1) to study health service process in the context of a system 
defined as broadly as it is herein, and 2) to use the classification and frequency 
of problems as the basis for a “universal” measure of performance has notrbeen 
tried. This proposal hypothesizes that there is a common ground betweenjdl 
observed defects in health service process (process problems). These process 
problems can be translated into a single unit of measure in terms of impact on 
the tfealth status of the individuals involved. By translating process problems in 
accessibility, those derived from coordination, and those resulting from 
improper medical judgment into estimated man-day equivalents of impairment 
and by estimating the total frequency of each class of such problems in the 
total performance of a health service system, a number of useful/purposes are 
served. Management can gain information on the specific areas /within which 
the performance of the system is deficient. Given limited resources, rational 
decisions can be made between alternative organizational changes which may 
be expected to relate directly to the effectiveness of the organization. 
Trade-offs such as between decentralization (increased accessibility) and 
coordination (a function of spatial contiguity) can be more accurately 
estimated. Additionally, the ability to estimate accurately the numbers of 
medical care prceess problems of a significant type and their aggregate impact 
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upon the health of the population to be served promises to yield an evaluative 
tool of considerable importance. 

The organization and management of personal health service systems 
presents problems not common to industrial management. The health service 
system is engaged in the production of a custom made product for each 
consumer. It does net have a homogeneous or standardized output. Each 
person who defines himself as ill is a single problem for impairment control 
drawn from a finite but vast set of manifestations, etiologies, and appropriate 
therapies. The paths 29 appropriate for diagnosis and management of care are 
too numerous to anticipate. Each medical management case is unique or almost 
unique. It is a complex of personal, social, technical and biological interactions. 

Tile work of the Program Area Committee on Medical Care Administra- 
tion of the APHA in classifying medical care process characteristics associated 
(presumptively or demonstrably) with good medical care has attracted 
attention to the deficiencies of evaluative systems based principally upon 
appraisal of technical quality. The intervening variables in accessibility, 
acceptability, continuity, coordination, comprehensiveness, purposiveness, and 
efficiency may result in gross loss of social effect, even where the technical 
quality of care is high. Medical teaching centers operating in the heart of 
deprived areas are often examples of such a paradox. 

Medical care process has been discussed by many observers. Ideally, the 
process can be described as follows, at least in terms of the system's 
predominant function of illness control: 

1. Individuals in the population take the illness role only when it seems 
necessary to do so-they must be'free to take the role when necessary 
and they avoid taking the role when there are other methods of 
behavior possible. Illness is both socially functional as a “cop out” 
mechanism and socially dysfunctional as a disruptive factor. The 
discrimination between appropriate referral to professional care and 
inappropriate referral depends upon knowledge. 

2. Individuals are capable of making appropriate judgment about the 
proper course of behavior for various illness states or for general 
health maintenance. At appropriate times, the ill individual himself 
will choose to seek professional care. In the absence of appropriate 
judgment by the individual, family and friends will refer appropriately 
to the professional system. 

3. There will be no psychological, social, or physical deterrents to 
appropriate use of professional health care (availability and accessibil- 
ity)- 

4. When the ill individual seeks professional care, the relationship should 
be one of mutua- acceptability for both the patient and the 
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professionals. r .ole expectations should not be breached in the 
process to the extent that patient-healer confidence is lost. 

5. The professional conduct of the healer should conform to standards 
of excellence developed under the criteria of science and humanism. 

6. The full range of specialized resources appropriate for dealing with 
the patient’s needs will be brought to bear in an efficient manner and 
coordinated for the maximum effectiveness. The criterion of effective 
care is the return of the patient to optimum health as rapidly as 
possible. 

7. The patier t relinquishes the illness role insofar as he is able as soon as 
he is able. 

It is proposed that the medical care process problems study will extend 
the process observations to those persons who do not enter the professional 
health service system as patients. An analysis will be made of the ability and 
performance of that part of the system to make proper judgments and to relate 
functionally with the professionals. Knowledge about appropriate health 
practice and caie seeking will be viewed as a function of the health care system. 

In addition to this extension of the concept of system, our observations 
will be extended to those who withdraw from the system (or from treatment) 
in order to locate process problems which may have prompted or been 
associated with such withdrawal. A patient, for example, who is advised to 
obtain dental service (not provided by the referring professional) may, from 
that time on, be lost to the record. Failure to get the needed care might lead to 
illness or disability treated outside the system after termination of member- 
ship. Without concerted effort to identify and quantify such process problems, 
there is little basis for appraising the effect of deficiencies in comprehensive- 
ness, continuity, or coordination. 

The Aggregate Health Impairment Durden Study: For the purposes of 
establishing a basis for outcome evaluation, the objectives of the health service 
system are deemed to be predominantly the exercise of control over the health 
impairment burden in the population. The health service system has other 
functions, as for instance the rendering of support, care and comfort in the 
normal process of childbirth and the validation of states of ill-health or 
wellness for insurance purposes, etc. Nevertheless, it is assumed that the 
predominant function of the health service system is in preventing, diagnosing, 
treating, rehabilitating from and mobilizing resources against actual and 
potential incidents of disease or sources of health impairment in the population 
at risk. The concept of aggregate health impairment includes all of those 
incidents of socially dysfunctional illness, pain, physical or mental impairment 
or incapacity which are health related. One goal of our research will be to 
develop methods for refined discrimination between socially functional use of 
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the illness role as a “cop out” and the socially dysfunctional impairment which 
causes use of the illness role. 

The urgency of exploring all promising avenues of end result measure- 
ment is stressed by Shapiro when he states: 

The desirability of determining quality of medical care by its effect on 
some measurable aspect of health is matched by the pessimism among 
researchers about the possibility of dealing with the issue. A reversal in 
this outlook seems essential today in view of the need to assess the 
meaning of the enormous changes in organization, financing and 
accessibility of medical services that are just beginning to emerge/ 

It is noteworthy that aggregate heal th impairment measures for the 
performance of comprehensive health service systems have not been systemati- 
cally explored, even though they promise substantial advantage. The use of 
mortality data as a measure of outcome for health service systems has been 
found to be relatively unsatisfactory. Except with the very young or with 
population segments especially at risk, e.g. pregnant women, mortality rates are 
not sufficiently sensitive to use as a basis for management. The relatively low 
incidence of death in most populations, even those receiving poor health care, 
makes it necessary that large populations be observed over long periods of time 
before the statistical credibility of the observed pattern can be established. The 
use of incidence or prevalence of selected disease categories is also problematic. 
These usually fail to give reliable indication of the general functional 
performance of the system and, due to low incidence, are subject to the same 
limitations as mortality data. In the search for an end result criterion which 
will be valid, reliable, and sensitive, the concept of an aggregate measure of 
health impairment incidents observed within a population is attractive-perhaps 
beguiling. 

Chiang , 30 Sullivan , 31 and Sanders , 32 > 33 have noted the utility of health 
impairment indexes. Shapiro notes with Sanders, the problem of using gross 
health impairment based upon incidence. In an ige ? when the miracles of 
modern medicine are increasing our ability to define peoule as ill while they are 
still able otherwise to function, it is natural that" better health service may 
increase the incidence of “illness.” Refinemenfqfjhe measure to embrace 
severity and duration of impairment will tend to overcome this problem. If 
there is any truth to our belief that prompt diagnosis and treatment will reduce 
disability, then a measure, of severity and duration of impairment should be 
sensitive to variations in the qualities of the health service system. Explorations 
in this field by Bush and Fanshel 34 seem promising. 

One further refinement of the measure of health impairment offers 
promise of improving the sensitivity of the measure as an evaluative tool. If it 
were possible to eliminate those portions of the health impairment burden over 
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which tiie health service system could have no control, the residual health ^ 
impairment measure would reflect those episodes which could presumptively, 
or hopefully, be affected by tl e health service program. Our research efforts in 
evaluation will seek to explore the usefulness of deveoping an aggregate 
measure of the health impairment in the population at risk. Through careful 
sampling techniques and the use of control populations experiencing health 
services in alternative health service settings, it is proposed that an attempt be 
made to translate all observable incidents of health impairment into an 
aggregate figure expressing incidence, severity, and at tribu tabllity in terms of 
man-day equivalents of disability. 

As an example of the process of assessing attributable (chargeable) health 
impairment in an individual case, let us take a person in the sample population 
who contracted venereal disease resulting (due to reinfection and other 
complications) in 37 days of health impairment before control was finally 
achieveG. If it were estimated that for five days he was 30 percent impaired and 
for the balance of 32 days he was 10 percent impaired, a severity corrected 
impairment score of 4.7 man-days of impairment would result : 



If it were further judged, in this case, that for the first exposure of five days’ 
impairment, 20 percent of the impairment was chargeable and that for the 
balance of the time 50 percent of impairment was chaigeable due to failure of 
tiie system to involve itself with the patient’s contact, the 4,7 man-days of 
impairment would be multiplied by a factor of 



The chargeable health impairment units in this specific case would be 4.7 x 46 
= 2.16 CHIMPUs (chargeable health impairment units). 

Such an approach allows for translation of the complete range of kinds 
of cases a health service system would be dealing with into a single uniform 
measure which might be standardized and applied in other health service 
systems for comparable data. 

The CH1MPU is not a measure of effect; it is a measure of the aggregate 
problem not controlled. A measure of effect could be derived by substracting 
the aggregate or mean CHIMPU score from some standard estimate of 
CHIMPUs expected in the absence of any health service (or from past 
performance levels). This yields an estimate of net increase or decrease in 
CHIMPUs for the current period. 



[(5 x .3) + (32 x ,1) = 4,7] 
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Methodological Concerns: The absence of prior research in aggregate 
health impairment measurement or in health service process as here conceived 
makes this proposed research essentially exploratory. A number of questions in 
End Result Measurement and Process Measurement are answerable only in 
theory until concrete data are generated: 

Can health impairment burden be measured? Can health impairment 
burden be demonstrated to be a valid measure of program outcome? Can the 
portion of health impairment which is equitably attributable to the medical 
care system be estimated? Can the processes of the program be observed and 
process problems be identified, counted, and weighted? Can the study 
population be effectively compared with other medical care delivery systems? 

Data for both the health impairment study and the health service process 
problems study will be gathered at the same time when appropriate. Also, the 
first two research components will be employed simultaneously. Data from 
each component is self-contained and can be analyzed alone. However, a data 
set will be assembled combining the data gathered by both methods. 
Subsequently, die laboratory component will be used for in-depth experi- 
mental treatment of process problems suggested by the data. It will also be 
used to initiate independent experiments with program structure alternatives 
In the following paragraphs, each of the major research components is 
outlined, beginning with a description of the sample on which components one 
and two are based. 

The Sample: A random sample of 800 persons stratified by age, sex, ana 
socioeconomic status will be selected from the population of those who enroll 
in the medical care program. From this sample a subsample of 100 cases will be 
selected for intensive study including a physical examination. (In subsequent 
panels, this 100 case subsample will be selected from those persons who have 
not received services since the last interview.) Retrospective data will be 
collected on the past medical care process and on health status. All respondents 
will be interviewed every six months for five years. Data from patient records 
will be gathered concomitantly with interviews. The sample size will insure 
that a large (lumber of respondents can be retained for analysis over the five 
years. To compensate for that attrition whicli will occur, continuous 
replacement will be employed. The original sample size and composition will 
be maintained by selecting new cases from the new enrollees. In this manner, 
the final sample will include 800 cases who have all been followed through the 
duration of their membership, though that membership may not extend the 
full five years. 

Control A: Comparative medical care process data from a system 
dissimilar to the principal program. Comparable health service process data will 
be developed on a sample of 400 cases who were eligible for the study program 
but who chose the Blue Cross alternative program. These cases will also be 
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interviewed semi-annually for five years. Continuous replacement will be 
applied to maintain a 400 case base. The same interview schedule will be used 
as in the principal longitudinal sample. 

Control B: Comparative data on health status from a similar system. A 
random sample of 160 cases will be drawn (stratifying again by age, sex and 
socioeconomic status) from new enrollees in a medical care system similar to 
the principal program. The comparative system must be similar in two respects: 
1) the population at risk must be definable and comparable to the principal 
program population and 2) the medical care plan should be basically the same, 
a pre-payment plan with voluntary enrollment. Health status data will be 
gathered annually through interviews and examination of records. Continuous 
replacement with new enrollees will compensate for attrition. This sample may 
be modified to meet needs dictated by preliminary data analyses. Process data 
will not be sought on this sample unless the medical care plan requests that we 
do so. 

Control C: Effects of measurement. Each six months a new sample of 
100 program members will be selected without replacement to be measured at 
the same rime as the longitudinal sample is measured. The objective in this 
control mechanism is to separate out the effects of repeated measurement for 
the continuing respondents. 

Control D: Data on bias in selection of population at risk in me study 
program. This last type of control will be provided by secondary analysis of 
university records and by comparative analysis of the descriptive data on the 
sample of eligible non-members described in Control A. 

The Interview: As mentioned, the interview will provide data relating 
both to health status and to medical care process. Essentially four areas are 
outlined in the basic interview schedule as it will be applied to the principal 
program sample and to the longitudinal sample of eligible non-members. A 
shorter interview related to health status will be organized for administration 
to control group B. A partial outline for the larger interview schedule appears 
below-the delineation of specific items to be included in the research 
instruments is itself considered to be a major part of the research effort. Items 
include: 

1. Background Data-Identifying Data, Socio-Demographic Information, 
General Attitudes Toward Medicine and Health Care, and Attitudes Towards 
Past Health Service System; These areas will be primarily covered in the initial 
interview although changes in attitude toward organized practice, preventive 
medicine, etc., overtime must be measured and would be important. Also it 
would of course he important to note changes in family, education, etc. A 
patient’s general attitude toward the system influences the whole way in which 
he relates to the system in each incident. Thus, if he feels generally negative, he 
may not seek help and a deviation from the ideal process will be noted. 
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The medical care system may function more or less well for members 
with differing socio-demographic characteristics and attitudes Unyard medicine. 
It is, of course, necessary to provide adequately for the control of such 
conditioning variables. The initial enrollment in the program may itself be 
affected by whether a person has chronic medical problems, a large family, 
financial problems, and the like. It will obviously be influenced by whether a 
person is satisfied with his present provisions for medical care and whether he 
is positively disposed to group practice. Once in the program, the member's 
attitudes will modify. his experience in the program. In addition to knowing 
how a patient's attitudes affect his experience, it is imperative to know if the 
program itself is structured to the advantage or disadvantage of members with 
specific attitudes and characteristics. Tor example, if all members with certain 
ethnic or age characteristics were dissatisfied, the program structure should be 
questioned. 

II. Recent Health Impairment Levels-ineludes respondent’s recall of 
specific impairment over a number of weeks to be established in pretesting; 
respondent’s recall of major impairments over the past six months; and 
respondent's perception of his current health status. Development of measures 
for the respondent's perception of extent of health impairment is a critical task 
in this research. As discussed in the previous section, a person may be more or 
less impaired by the same incident, depending upon his normal roles. Thus, 
measures’" must be developed to allow a respondent to assess his relative 
impairment with respect to the major spheres in which he functions. This may 
include his relative ability to maintain normal function in the family, school, 
work, social relationships, both formal informal. 

III. Data on the Process by Which Medical Care is Administered and 
Received-inciudes action taken by respondent in response to each health 
impairment incident and reason given by respondent for taking the action, etc. 

Although the respondent may not recognize some difficulty he has as the 
fault of the medical care system, there may be in fact a connection. For 
example, in a health care study in Atlanta, Georgia, it was found that 43 
percent of the respondents worried about bothering the doctor. A patient 
might conceivably realize he let something get out of hand because he didn't 
want to bother the doctor. He might not see this as the fault of the system, but 
a system breakdown might, nevertheless, be occurring. 

The interview schedule will be repeated every six months. Specific 
additions will be made as they are suggested by the development of the health 
care program. Should a great number of respondents raise a certain issue, that 
issue might be explored in the subsequent panel. 

All interviewing will be done by a professional interviewing staff which 
matches the distribution of the patient population in age, sex, and socio- 
demographic characteristics. Interviewers will be selected who are not related 
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to the heal tli service system in any other capacity. The interviews will he about 
an hour in length and will be administered outside of the clinic setting. 

Overall estimates of the health impairment level of the population will be 
made after each panel. Estimates will also be made as to incidence of certain 
types of failures to reduce or avoid impairment and of the incidence of certain 
* classes of process problems. Working with a panel of medical experts, it should 

be possible to develop a set of criteria that, from vhe interview data alone, will 
allow detection of some failures or problems. For example, if the respondent 
cites inability to get an appointment as a reason for not getting to the clinic, 
this can automatically be counted as a process problem. From this type of data 
the point at which system breakdowns are occurring can be identified for 
further analysis. The interview in question would be paired with the clinical 
record and referred to the medical panel for analysis and classification of the 
problem. 

Secondary Data tram Records: Data to be gathered from the records will 
include a draft of data available from the management information system and 
a complete coding of other informatic from records belonging *o sample cases 
and internal control cases. 

Data will be gathered from both clinic and related hospital records in 
conjunction with each interview panel. Ail records for the 800 membership 
sample cases will be coded each six months at the time of the interview. The 
records of internal control cases will be coded only at the time that they are 
interviewed. It is probably not possible to do similar coding for non-member- 
ship cases although alternatives should be explored. * 

A coding scheme will be organized in the same manner as the interview 
schedule is organized. Each six month period, the record will be examined for 
specific information which would include, for example, for each incident 
length and amount of impairment associated w ; th the incident, course of 
action prescribed, etc. Exact items of data to be coded will be delineated as the 
methodology is developed. 

Professional Panne! : As coding proceeds, all cases in which process 
problems are possibly operating will be designated for special consideration in 
the review by the medical panel. The functions of the medical panel, which 
includes three physicians as well as other experts in the medical care process, 
will be to determine for each case in the sample: the degree of gross health 
impairment, the degree of attributable health impairment, presence or absence 
of health service process problems, classification of such problems, apparent 
cause of such process problems and health impairment associated with such 
problems. 

A comparison of data gathered in this manner from secondary sources to 
data gathered through interviews, yields first of all an itemization of data 
which can be reliably gathered from secondary sources. From the above it is 
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then also possible to obtain a second estimate of the level of impairment of (lie 
population. This estimate must be compared with that estimate derived from 
the interview data. The degree to which these estimates converge has obvious 
and far reaching implications for the remainder of the data analysis. 

A comparison of the record data and the interview data on the simplest 
level will provide an estimate of the proportion of health impairment incidents 
which appropriately reach the attention of the medical care delivery system. 
From the records alone it is possible to judge the proportion of patients for 
whom the appropriate medical action «va. 1 taken and for whom the appropriate 
follow-up occurred. Comparing the recc rd data with interview data allows 
more sophisticated judgments with respect to the appropriateness of treatment 
in the population at risk. If the interview and the record usually agree on the 
level of success with which incidents are treated, direct estimates of the clinic’s 
effectiveness might be made with some accuracy from the professional record 
alone. That is, wc can estimate the average impairment suifered by the 
population, the percent of incidents in which the clinic was consulted, the 
responsiveness of the clinic, the cooperativeness of the patient and the success 
of treatment. Further, specific points at which the system is apparently prone 
to failure can be identified for further study. 

When the case record and the interview conflict, that in itself may 
indicate a system breakdown of some nature. A specific problem may be 
suggested by the character of the disagreement. A more expert analysis of the 
records and interviews in question may point to the source of the system 
failure. However, it may be that an area for further research is indicated. The 
application of subsequent interview panels will be guided by such indications. 

The Laboratory : The labotatory component can be used ip many ways. 
Among its principal uses will be the follow-up of process problems or 
measurement problems which are identified in the preceding data gathering 
efforts. The content of the interaction process which seems to be creating a 
system breakdown may be better understood through direct observation or a 
related experimental technique. Through the design of specific experiments, it 
can be learned how the system responds to particular kinds of illnesses, patient 
personalities, age or sex categories which have been found to be associated with 
unsatisfactory outcomes. 

Additional specification of laboratory research can only he done as 
preliminary data emerge from the management system and from the 
longitudinal study of patients. 

Analysis: Two levels of analysis will be required in this research. First, 
some evaluation of the effectiveness of the principal program must be made. 
Tills involves the estimate of the impairment levels in the population at risk as 
they change overtime and as they compare to impairment levels in other 
populations. It also involves the incidence of process problems in the medical 
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care system, and the relationship of process problems to health impairment 
outcomes. 

The second level of analysis pertains to <he evaluation of, and refinement 
of, this methodology. This analysis requires intensive and continuous inter- 
action with the data. It is first of all necessary to test the validity, reliability, 
and sensitivity of health impairment burden as a measure of outc< me. Second, 
the assessment must be made of the accuracy with which attributability can be 
measured. The criteria which emerge fiom the medical panel for assigning 
attributability must be tested for validity and practicality. 

The project will require the continuous support of a coding and 
programming staff. All data will be coded for computer analysis and a system 
will be instituted whereby successive data analysis requests can be made. The 
computer facilities at the Health Sciences Computer Center will be used. A 
programme! will be employed who is specially trained in the application of 
multivariate statistical techniques. 

Since this is an ongoing evaluative effort, it would be inappropriate to 
think of data analysis as the testing of a pre-determined statistical model. 
Relevant statistical techniques will be applied as the preliminary analysis 
dictates their desirability. 

Vic Significance of This Study: The LCLA Center for Health Sciences 
has, along with many other University centers throughout the nation, 
e\perienced an increasing interest over the last five years in the problems of the 
delivery of health service. Excursions into regional medical programs, con- 
tinuing education, and relations with comprehensive health planning endeavors 
have led to the realization that much of the technological capability of 
medicine is not being realized due to structural-organizational problems in the 
world of medical practice. Concurrently, changes in the nature of the 
population served and the teaching of health professionals have given rise to an 
awareness that centers such as ours must not only anticipate changes which are 
going to dominate the medical world of tomorrow, but that they must use 
the'r resources to provide the professional leadership necessary to assure that 
the changes which do occur are going to be favorable for society. 

Stimulated by the creation of California Center for Health Services 
Research, the UCLA Research Program in the Organization of Health Service 
has made its resoutces available to the UCLA Hospitals and Clinics to assist in 
the planning and development of a number of model health care delivery 
programs. Each of these programs is conceptualized as a laboratory within 
which experimentation in the organization of health services can be undertaken 
to determine its effect upon health outcome and on the efficiency of the 
health service delivery system. The imperfect state of the art of health services 
evaluation, however, makes it imperative that, at the outset, a considerable 
thrust be made in order to develop the criteria for assessing the effects of the 
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system. Both a multidimensional process evaluation technique and a practical 
end result measurement are imperative to the success of these laboratories in 
yielding definitive information on the effectiveness and efficiency of alterna- 
tive health service patterns. Both the medical care process problems study and 
the aggregate health impairment study are believed to be the most likely 
methods by which the model programs can be definitively evaluated, and by 
which sufficient management information about health service process and 
program effectiveness can be generated to allow for fulfillment of the purposes 
of the program. 

One major side-effect of the effort to develop an evaluative methodology 
will be the contribution of this study toward the development of a new 
approach to management information systems in health service delivery. The 
surveillance of health service process problems will provide, we believe, a 
radical new tool for health system managers. 
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PART II: 



AREAS OF EVALUATION 

PROBLEMS IN DEFINING TARGET POPULATION 

Jacob J. Feldman, Ph.D. 

By contrasting two rather disparate views of the responsibilities of the 
medical profession, the concept of a target population may be somewhat 
clarified. 

• Medicine need be concerned only with those cases of disease and injury 
which are presented to physicians. The major objective is to provide 
the best treatment the present state of medical knowledge permits for 
those cases. This view tends to arise out of the pressures of day-to-day. 
practic, of medicine rather than, from ideology. Many find the 
challenge of managing the morbidity' which is brought before the 
profession sufficient to occupy their total attention. This is the 
traditional clinical v iew. 

• Limited medical resources have to be .o allocated as to maximize the 
welfare over the total population, even at the price of sometimes 
providing less than optimum care for an individual patient. Prevention, 
early detection and early treatment are the responsibility of medicine. 
Outreach activities are essential to bringing health care to individuals 
who do not freely seek it. Here we have the view point of community 
medicine and public health. 

The notion of a target population is not compatible with the clinical view 
of medicine. As specialization becomes more intensive and_practice more 
fragmented, it is difficult enough for the practitioner to maintain a holistic 
view of the individual patient, let alone feel concern for a total population. The 
major criteria by which performance is assessed are cure rates or survival curves 
for attended cases. A conventional solo or group practice is generally not held 
responsible for the deterioration of a potentially manageable condition for 
which no care was sought or for which care was sought elsewhere. Similarly, 
clinicians are generally not blamed for the incident of potentially preventable 
illness, whether it is environmentally induced or the result of the afflicted’s 
own behavior. Such limited accountability is a concomitant of the free-for- 
serviee payment mechanism, regardless of whether the fee is paid by the 
individual consumer or by some third party. 

The introduction of prepayment into a group practice alters the 
situation. By contracting to provide the medical care required by an individual 
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a specified time period, the group practice may be viewed as assuming a more 
general responsibility for the health of that individual. Thus, the enrollees in a 
prepaid group practice plan can be considered a population. Rates charac- 
terizing the health status and medical care utilization of sucli a population can 
be taken as, in part, reflecting the performance of the H iOup practice. However, 
there may be certain difficulties in using these rates in the appraisal process. 

• Usually, little information is available concerning characteristics of the • 
enrolled population. This is currently being somewhat alleviated by the 
collection of rather detailed information pertaining to all family 
members at the time of registration. However, changes in family 
composition and various ocher characteristics occur frequently. Few 
plans have taken the trouble to keep current the data concerning their 
populations, even on a sampling basis. 

• Enrollees are not generally captive populations. Some seek care outside 
the plan and considerable morbidity goes unreported within the plan’s 
records. In a few instances, special interview surveys have been 
conducted to collect such supplementary information. However, these 
are expensive operations and have been conducted only rarely. 

• In some statistical systems, the basis of tabulation is the individual unit 
of service (visit to the physician or nurse, hospital admission, 
laboratory procedure, etc.) The discrete encounters are not linked 
together or cumulated either in terms of episodes or individual 
enrollees. Under such a system, it is easy enough to count the 
aggregate number of visits of a particular type, compute the mean 
number of visits per person enrolled in the plan and even produce 
service rates specific to age-sex or similar groupings. What are missing 
are, for example, distributions by number of patient contracts during a 
given period, duration of interval between patient contacts and use 
patterns across different types of care. From aggregate service statistics 
we can learn about the workload of those providing the services. 
However, the cumulative services of a series of individuals over a^ttme 
period must be reviewed if we are to assess the appropriateness of the 
allocation of staff effort among enrollees. If, as suggested earlier, a 
prepaid group practice assumes responsibility fpr the health care of an 
enrollee whether or not that enrollee actively seeks care, then we must 
have tabulations for which the enrollee is the statistical unit. 

The organizers of the present conference have restricted participation to 
“representatives from university medical centers which provide medical care to 
a defined population.” By relating the services provided by a medical facility to 
the population for which that facility is considered responsible, we have a more 
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meaningful measure of process than if we had only service statistics. The 
“defined populations” supply the denominators for our rates. In this 
connection, we frequently need be concerned by the representativeness of our 
population or, at least, the comparability of the populations designated as 
targets by different plans. One of the chief purposes of each of the programs 
represented here is the promotion of scientific knowledge concerning the 
organization of medical services. In addition, even for. purposes of internal 
management, utilization rates are interpretable only in comparison to the rates 
which have been experienced under other sets of circumstances. Since 
utilization rates are appreciably influenced by population characteristics, it is 
extremely important to determine how the various populations differ from 
each other. For instance, we need to know the extent to which the enrollment 
procedure of a given plan selects in favor or against families with high 
morbidity rates or a propens'ty to use medical services. Clearly , restricting one’s 
target to an enrolled population does not permit one to ignore the more 
general population from which the enrollees were drawn. 

While some university medical centers seem to be dealing exclusively 
with an enrolled population, whatever its origins, other centers treat the wider 
community as falling within the purview of their programs. Some of the plan 
administrators would like to treat as their target population all residents of a 
geographically defined catchment area. Such a broadly defined population 
creates, under present financing and organizational practices, several problems: 
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In situations where the operation of the center is to be supported 
primarily by public funds, a means test is frequently mandatory. 
Thus, it is necessary to have information concerning income, family 
size and perhaps additional financial information for each of a sample 
of families in order to estimate even the size of the eligible target 
population, let alone its characteristics. Unfortunately, adequate 
family financial data for a catchment area are only rarely available. 
Even if a survey had been conducted in the recent past, it is not likely 
that the resulting data would be useful in estimating the number of 
people eligible for services. The survey questions would have needed 
to have been so formulated as to elicit information conforming 
conceptually to the eligibility guidelines under which the center 
operates. Thus, something as simple as a count of eligible residents 
requires the initiation of a rather complex data collection operation. 
It is sometimes implicit in the inclusive definitions of a target 
population that it is an objective of the center to provide care for all 
eligible residents of the catchment area. This goal may well be 
unreasonable. At the time of the inception of services, some members 
of the population will already have a satisfactory source of medical 
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care. It would be difficult to justify interference in an established 
medical relationship. Thus, in gauging success in terms of target 
population penetration, it would be desir?: *' "o exclude from the 
denominator those individuals who already ,ave available services 
comparable to those being offered by the center. While it may 
sometimes be possible to draw inferences from operational records 
concerning alternative sources of care for certain limited segments of 
the population such as those covered by Medicare, more usually such 
information must be collected by interviews with a sample of area 
residents. 

3. The problem of estimating the size of the target population is 
complicated further by the instability of the characteristics at issue. 
As unemployment increases one or two percentage points nationally, 
the impact on sensitive lower income neighborhoods can be a great 
deal larger. The size of, the pool of families eligible to receive 
subsidized care may increase substantially. If one considers in 
addition the effects of migration differentials, changes in the 
regulations governing eligibility and changes in the accessibility of 
alternative sources of care, it is clear that practically continuous data 
collection is required if estimates of the size of the target population 
are to be current. 

It is clearly expensive to collect the data required for the relatively 
accurate estimate of the size of a broadly, but realistically defined ^rget 
population. While the enrolled or registered population provides appropriate 
denominators for many of the utilization rates in which one interested, a 
plan with a commitment to outreach as the foundation of its program would 
derive valuable feedback from penetration rates based on the components of its 
actual target population. Without a substantial investment in household survey 
data, community based penetration rates are often subject to such large errors 
as to make their use dangerous. On the other hand, a cost-benefit analysis may, 
in a particular instance, show large scale survey operations to be an 
uneconomic expenditure; alternative demands on limited resources may 
preclude adherence to high standards of survey methodology. My personal bias 
leads me to suggest that frequently even a small and relatively inexpensive 
systematic survey may well serve to correct widespread misconceptions 
concerning a particular population. I am perhaps unjustifiably skeptical about 
the accuracy of quantitative estimates derived from expert judgment rather 
than data. 
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DISCUSSION ON 

PROBLEMS IN DEFINING TARGET POPULATIONS 



Chairman : Sidney S. Lee 



In the discussion concerning the general concept of the target population 
serviced by the university medical care plans two fundamental questions were 
asked; what population does the plan attempt to serve; and why was this 
particular population selected? The consideration of these questions generated 
considerable discussion. On the one hand, the participants stated that health is a 
social utility and therefore health care is a fundamental right of all persons in 
the community; on the other hand the university groups had to contend with 
varied demands and problems which limited the selection of their target 
population, and thus the number of persons the plan could serve. It was also 
noted that no health plan, of whatever nature, can take care of the total 
population of an area, and that university groups could not assume 
responsibility for blanket coverage of large populations. 

In those plans which serve inner city poverty populations, such as the 
Community Croup Health Foundation, Inc., or the Matthew Walker Health 
Center, the geographic boundaries themselves, as well as income level, limit the 
provisions of the services which can be offered. The same restriction is inherent 
in other olans where the provision of services is by definition limited to; 1) 
those wh< can purchase the plan; or 2) those who meet a regulatory demand. 

From the viewpoint of the patient, it is practically impossible to set 
boundaries for eligibility, be they geographic or regulatory, that will satisfy 
everyone. There will always be people in need who live across the street from 
an established geographic boundary or who are prepared to travel great 
distances to obtain care at some particular facility. Undetermined proportions 
of the population will be “borderline ineligibles” for various reasons; such as 
family income level; or federal regulations when the government is the third 
party; or specific contract coverage provisions when a private insurance 
company plays the role. The university groups were acutely aware of these 
problems and expressed concern over the necessity of limiting their plans to a 
segment of the population when the people excluded could not be referred to 
an alternative source of care of equivalent quality. 

Before beginning to function, each plan was encouraged to: comply with 
laws or regulations affecting incorporation of financing; assure third party 
payment; use existing insurance carriers for marketing plan; simplify the 
enrollment process for prepaid group insurance; permit the selection of study 
and control groups for research; obtain a desired blend of middle and low 
income enrollees; meet the request of organized community groups asking for 
service: set a ceiling number which conforms to a cross section of the total 
community population. 
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Clearly, the realities of marketing and financing the university medical 
care plans put constraints on the populations to whom the service benefits 
can be offered. It was agreed, therefore, that the target population should be 
defined as the population, eligible by a variety of economic or geographic 
criteria, to which the plan will offer its service benefits. Thus the target 
population is essentially the “market” which the plan would like to reach. 

The following table presents a brie r summary of the discussions of the 
target populations of different plans. 

Additional discussions about definitional problems relating to the 
population brought out the need to determine the relationship among the 
target population, the elements within that popu^tion who are “at risk of 
receiving service,” and those individuals who are actually served by the plan. In 
an epidemiological sense, the enrollees in the prepaid plans would be “the 
population at riskr of receiving services”; in the neighborhood health centers 
“the population at risk of receiving services” would be the registrant 
population. 

REACHING THE TARGET POPULATION 

As shown in the Table, the requirements for entry into the medical care 
plans varied but, in general, included one or more of the following: 

1. Enrollment in the plan through prepaid group insurance, usually 
related to company employees, union members, or other organized 
groups within a defined geographic area and requiring a contractual 
relationship to receive the service benefits offered by the plan; 

2. Registration of persons from geographic, low income area usually 
related to persons who meet neighborhood health center eligibility 
guidelines; registration in such a plan is through a household canvass, 
survey, or a family member seeking care at the center; service benefits 
are available to all persons who meet eligibility requirements and live 
in the specified area; 

V 

3. Eligibility to receive care through Title XVIII, Title XIX, or other 
government programs usually related to agreements between the 
medical care plan and the state department of welfare (or other 
appropriate official agencies) which enable the plan to provide service 
benefits on a per capita basis to a limited number of persons in the 
appropriate eligibility category; the limits can be set by specific 
number of persons and/or the geographic area. 

Discussions centered on the difficulties faced by the plans in trying to 

assess the degree to which their target population are being reached. It was 
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pointed out that the mode of entry adopted by the plans is a major factor in 
determining not only the target population to be served but also the method 
lor evaluating the degree to which the needs of that population are met. 
Except for the enrollment process, the method of entry may present serious 
difficulties in obtaining information about those segments of the target 
population which are at risk of receiving service but not served- a critical issue 
in the evaluation ol how well the plan meets one of its prime objectives, to 
improve the health of the target population. 

It was observed that any estimation of the size of the target population is 
complicated by the instability of the characteristics at issue. As unemployment 
increases, perhaps only one or two points nationally, the impact on sensitive 
lower income neighborhoods can be substantially larger. The number of 
families which are eligible to receive subsidized care may increase significantly. 
There are also factors of migration differentials, changes in the regulations 
governing eligibility, and changes in the accessibility of alternative. sources of 
care. Thus it is clear that some type of confining data collection is required if 
estimates of the size of the target population are to be current and accurate. 



SOURCES OF DATA AND CHARACTERISTICS OF THE POPULATION 

All of the university groups obtain' demographic data on those persons in 
their target population who choose to enter the plan. These data are collected 
at the time of enrollment for all persons who enroll in one of the university 
prepaid group plans. Thus there is a clearly defined “population at risk of 
service” for the prepaid group plans. In such plans as the neighborhood health 
service centers, data are collected at the time of registration which usually 
occurs at the time a person first seeks medical care and completes a registration 
form for himself and for other members of his family. Thus because of the 
manner of registration and the lack of contractual relationship there are large 
segments of the population for which there are no data, and therefore, no 
clearly defined “population at risk of receiving service,” In an attempt to 
minimize this problem, some of the plans conducted household canvasses to 
pre-register persons eligible for service; other techniques include mail surveys, 
household sampling, and general populatior surveys. But the basic point at 
issue in all the plans is that a continuing relationship has been established 
between the plan and the target population and that therefore a determined 
effort should be made to further this ongoing process. 

Both Dr, Feldman and Dr. Elinson discussed the many difficulties 
involved in such endeavors. As one example of the difficulties, the Community 
Health Foundation, Inc, noted that the usefulness of their population survey 
data is limited; due to the high mobility of the young adult, black male and to 
the constant changes in eligibility requirements. However, current data from 
surveys being conducted by the East Baltimore Medical Program and by the 



87 



o 

LIf 



82 



Columbia Medical Plan indicate population mobility will not be a very grave 
problem. Household surveys are being conducted in each of these communities 
in order to obtain demographic data on the target populations of each of the 
plans. 

The Community Health Care Center Plan will attempt to enroll a 
population corresponding to a cross-section of the total population with a 
possible higher representation of low income families. Currently , baseline data 
are being collected to be used for pre- and post-enrollment studies of their 
target population. For comparison, these data include an appropriate sample of 
the total population of the greater New Haven community. The Matthew 
Walker Health Center is attempting a “head count” census in its geographic 
area to obtain data on the total population and on the target population. 
During the interviews, the Center attempts to discover the number of people 
who will use the Center as their source of health care. 

The demographic characteristics of the populations to be served by the 
universKy medical groups show great variation. With respect to the enrollment 
populations there are attempts to obtain a 20-30 percent enrollment of low in- 
come groups through per capita or other prepaid mechanisms. By contrast, plans 
with registrant populations contain an overwhelming majority of persons from 
low socio-economic groups. Age, race, and sex characteristics also varied 
markedly among the different plans. 

In addition to the current population data being collected, the university 
groups discussed the need to develop methods for keeping up with the changes 
not only among the “population at risk of receiving services” in their plans, but 
also in their target population. The prepaid group plans did not consider the 
up-dating process a major problem since those who enrolled were covered by a 
contract which terminates if the enrollee leaves the plan. The dual choice 
option of some university prepaid plans offers a method for obtaining data on 
Uie target population, since it is possible to obtain data on those persons who 
selected either plan. All agreed that the problems facing plans with a registrant 
population are enormous, and although there was much discussion, no 
solutions were underlined. The use of sample surveys, telephone or mail 
surveys and other techniques were among the methods being considered. 
However, Dr. Feldman expressed his skepticism about the accuracy of 
quantitative estimates derived from expert judgment rather than data, and he 
said that without a substantial investment in household survey data, com- 
munity based penetration rates are often subject to such targe errors as to make 
their use dangerous. 

MEASURING ACHIEVEMENT 

Since most of the plans vary in the criteria for selecting their target 
population, in the priority of their objectives, and in the characteristics of their 
target population, any attempted comparisons among the plans should be 




88 



83 



viewed with utmost caution. A|thc\.gh eacli of the plans is seeking Ways to 
measure its impact on its targe populations, the groups agreed that each, of 
course, lias to establish its own criteria for measuring achievement. Yet, 
beyond the question of performance, Dr. Elinson pointed to the need for the 
“outside observer" to have some framework within which to evaluate the 
objectives of the plans. Thus he suggested two approaches: I) an evaluation of 
the plan’s progress in achieving its objectives; and 2) an evaluation of the social 
utility of the objectives. The latter approach would require the identification 
of some non-quantitative criteria for assessing the degree of social utility in the 
objectives of the different plans. A discussion ensued concerning the difficult 
judgments which had to be made in order to assign values to each of the items 
in individual plans. For instance, in the area of involvemen 1 , a judgment has to 
be made about community control as opposed to university control, and in the 
area of funding, a judgment between grants and insurance; in the area of 
objectives, between service to a large number of people and training and 
research; in the area of responsibility between a determined enrollment 
campaign and a laissez-faire attitude where enrollees are accepted only when 
they happen to apply. 

During this review, the university groups began to delineate similarities of 
purpose and oiice more to focus on ways to make comparisons among the 
plans. The participants felt that, to the degree that the medical care plans share 
a common purpose, it should be possible to develop information which could 
permit comparison of utilization and costs. Thus the capability of the plans to 
make such comparisons depends upon similarity in the target populations and 
in the determination of the “population at risk of receiving service” during a 
specified unit of time. 

A major conclusion emerging from this session was that coming to grips 
with the issue of target populations is a difficult but crucial task. Yet it is 
essential that the services provided by the medical care plan be related to its 
target population in order to determine the significance of the health care 
programs in the total fabric of services in the community. 
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OPENING COMMENTS 

ON EVALUATION OF MEDICAL CARE PROCESSES 



Joseph Axelrod 



The charge for this critique was to appraise the “design of record systems 
which will yield criteria to serve as indices for characteristics of the defined 
population, utilization of services, attitudes of enrollees and staff and 
behavioral changes” of seven health center plans, each associated with a 
university. 

Reviewing the material submitted by these programs, it is evident that 
their evaluation plans in terms of medical care processes are not completely 
stated. In some cases plans for evaluation may not yet be fully formulated, in 
others they may only have been omitted from the submitted texts, and in some 
there probably are real gaps. Tables 1 and 2 attempt to reflect the stated 
methods for collection of data and evaluation of utilization, attitudes, behavior 
and satisfaction. Table 3 relates to research and teaching. The blank spaces 
indicate those elements of information not specifically mentioned as included 
in the evaluation plan or which cannot be inferred from other information. 

What we see is a pattern of strict attention to utilization and limited 
concern with evaluation of attitudes, behavior, compliance, and satisfaction of 
patients. In some of the medical care plans only termination of membership 
will alert the plan to problems in these areas. Four of the plans fail to mention 
provision for studies of broken appointments, non-utilizers, compliance with V_/^\ 



outside referrals, and compliance with prescribed regimens. Only two of the 
programs state methods to be used for complaint review and studies of patient 
satisfaction, although it would be difficult to believe these will not be 
established. In only two plans is the intent shown to report frequency 
distributions of enrollees by number of physician and/or medical services 
received. These data are often valuable indicators of the need for better 
communication with patients, more appropriate referral patterns, and specific 
health education programs. 

Compliance with prescribed referrals, diagnostic services, therapeutic 
regimens and follow-up are the stated objects of recording of only three plans. 
This is not a simple process, and one can be bogged down in methods 
attempting to attain 100 percent compliance. A recommendation which is 
realistic in this regard would be limited to sample studies and spot checking. If 
prescriptions for medication were all filled at the plan’s pharmacy, prenum- 
bered prescription blanks could be checked occasionally for compliance. At the 
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TABLE 1. EVALUATION OF THE MEDICAL CARE PROCESSES BY UNIVERSITY 
GROUPS AS INDICATED IN PRELIMINARY PLANS - APRIL 1970 



Utilization 


Harvard 


Hopkins 


Howard 


UCLA 


Wash 


Yale 


HCHP 


CMP 


EBMP 


CGHF 


CHCPP 


MCG 


CHCCP 


Number of physician services* . 


X 


X 


X 


X 


X 


X 


X 


Number of other professional 
services** 


X 


7 


? 


X 


X 


X 


X 


Number of hospital 

admissions*** 


X 


X 


X 


X 


X 


X 


X 


Number of E.C.F., etc. 

admissions*** 


X 


X 


X 


X 


X 




X 


Number of preventive services** 


X 


X 


X 


X 




X 


X 


Frequency distribution of 
enroUees by number of 
physician services 


X 












X 


Plans for comparison with other 
populations 


7 


X 


X 




X 


X 


X 



*By : Unit of time, type of physician, place of service, appointment/non-appoint- 
ment, and patient characteristics. 



**By: Unit of time, type of service, place of service, type of case, type of tests, 
and/or procedures. 

** ‘By: Unit of time, length of stay, patient characteristics, discharge diagnosis. 



least, a roster of patients with proved, life-threatening diagnoses requiring 
follow-up should be checked at intervals to identify the noncofhpliant for 
special attention. 

A form is employed in each plan which serves as a route slip, disposition, 
or encounter record intended to document each “in person” contact with the 
plan. Generally, these forms are very comprehensive in terms of the 
information recorded. In addition to the suggestion that the terminology in the 
encounter forms could benefit from standardization, recommendations follow 
which would make the forms more useful: 

• Although it is not indicated in any case, it would seem to be valuable 

to pre-number these forms so that spot checks of total reporting can be 

made. 
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TABLE 2. EVALUATION OF THE MEDICAL CARE PROCESSES BY UNIVERSITY 
GROUPS AS INDICATED IN PRELIMINARY PLANS - APRIL 1970 



Attitudes and Behavior 


Harvard 


Hopkins 


Howard 


UCLA 


Wash 


Yale 


HCHP 


CMP 


EBMP 


CGHF 


CHCPP 


MCG 


CHCCP 


Plans to measure behavior, 
















attitudes, satisfaction: 




X 




X 


X 


X 


X 


a) Broken appointment studies 








X 


X 




X 


b) Non-utilizer studies 

c) Use of non-plan doctors 








X 


X 




X 


studies 








X 


X 


X 


X 


d) Complaint studies 

e) Transfer/ termination 








X 


X 






studies 


X 


X 


X 


X 


X 


X 


X 


0 Consumer participation 
















effectiveness studies .... 








X 








Can the record system obtain: 
















a) Number of outside 




i 












referrals and compliance . 








X 


X 




X 


b) Proportion of patient 
















compliance with 
prescribed regimen 








X 


X 




X 


c) Barriers to utilization 










X 






d) Patient satisfaction/ 










- 






dissatisfaction with 
















5 crvices 








X 


X 






e) Patient satisfaction/ 
















dissatisfaction with 
clinic operations 








X 


X 






0 What will be the frequency 
















of data tabulation? 


mo. 


mo. 

























TABLE 3. EVALUATION OF THE MEDICAL CARE PROCESSES BY UNIVERSITY 
GROUPS AS INDICATED IN PRELIMINARY PLANS - APRIL 1970 



Research and Teaching 


Harvard 

HCHP 


Hopkins 


Howard 

CGHF 


UCLA 

CHCPP 


Wash 

MCG 


Yale 

CHCCP 


CMP 


EBMP 


Description of research 
















planned in delivery of care . . 


X 


X 




X 




X 


X 


Description of teaching link, 
















and whom taught 


X 




X 


X 




X 


X 


Teaching program financing . . . 


' y 


X 




X 








Teaching program evaluation . . 


( 

.. i 






X 
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• Day of the week, date, and time stamps would also prove valuable in 

■ relating onsets, first contacts, workups, counsultations, treatment, 

• disposition, and follow-up intervals, as well as to provide information 
from which traffic flow studies can be made. 

• Somewhere on the form there should be a place for the appropriate 
person to affirm that, to the degree possible, the diagnosis, treatment, 
follow-up and/or health maintenance plan recommended for the 
patient was communicated fully to him. 

• The same form should be able to be used to record home and inpatient 
contacts with plan staff members. 

The “defined population” of each plan requires clear delineation as well 
as standardization of terms. Here each plan should work toward identifying its 
population by some enrollment process linked to establishment of a permanent 
identity number. From these data the denominator of persons-at-risk by age, 
sex and family relationship can be established as a basis for utilization 
reporting. 

Effective and efficient delivery of medical services are common goals of 
all the plans represented. Yet, some plans have apparently not included testing 
attainment of these goals either by regular feedback or based on special studies. 
Mere compilation of utilization data and their analysis will fail to supply causes 
and possible correctives to the important questions concerning broken 
appointments, the non-utilizer, the use of outside obstetricians, surgeons, and 
hospitals; nor will patient compliance and satisfaction be evident. 
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CRITIQUE ON EVALUATION OF MEDICAL 
CARE PROCESSES 

Kerr L. White, M.D. 

Results are what count in patient care. Ends, not means, are the criteria 
by which we are judged both by our patients and profession. To the extent 
that examinations of processes explain variations in outcomes, they are useful 
exercises. To the extent that such examinations generate statistics activities 
alone, they are useless, if not harmful. They may detract from the important 
issues and are certainly expensive. 

A critique of processes isolated from their relationships to objectives and 
outcomes is virtually impossible. The objectives of all six university medical 
care plans, however, strike me as vague and obscure. Only two offer any 
statements that suggest that serious thought has been given to their objectives in 
relationships to patients’ needs. The UCLA plan states as its first objective: “to 
influence favorably (reduced) incidence and severity of health impairment 
(aggregate) in the population (insofar as it is technically possible and 
economically feasible).” The Howard University Plan has as its first objective: 
“to intervene in the poverty cycle through improving the health conditions of 
the residents of the Upper Cardozo community by providing better health 
care.” 

For the rest, their objectives deal with processes, mechanisms of care or 
with matters of interest largely to the professionals or the administrators. 

The operative phrases in these two statements are: reduce, incidence, 
severity, impairment, intervene, improving and conditions. The emphasis is on 
populations defined first geographically and second by enrollment. It is not my 
assignment to elaborate on measures of end-results or outcomes, but it will be 
necessary to refer to them in order to illustrate the utility and the relationships 
of these to process measures. 

If we are to discuss populations, we can express our results in rates that 
are related to either general populations or enrolled populations as the 
denominators. Age and sex standardization, to a standard population, would 
permit comparison between plans. I suggest that all measures for the six plans 
be expressed first, in relationship to their enrolled populations; second, in 
relationship to individual physicians or patient care teams; and finally, in 
relationship to patient care problems. Data about visits “made,” patients 
“seen,” x-rays “taken,” tests “done,” diagnoses “established,” injections 
“given,” and pamphlets “passed” are useless, in my judgement, unless they are 

89 



0 



94 



rchucd to the population served, to the individuals (professionals) giving the 
care or to the problems or conditions presented by the patients, and of course 
to the end-results of care. * 

The most useful clinical documents for our present purposes proposed by the 

fh^cVie t.l M, Cnt hnC T ter Forms; their valuc is enormously enhanced to 
extent that they are problem oriented.”’ I like the Hopkins’ encounter 

form; parhcuiarly smee our group has been deeply involved in Its development! 
TlUC ls P robdb| y too much information recorded, but it is a much 
*, h “ " ie or 'Sinally proposed. A simple s.aiemcn, of dm 
pj' on s eomptami or problem is lire slatting point, the classification scheme 

i rf fn, r S, ”“ ldin T g , ! ", a S,ar '- b "‘ " WHI . O .I 

ba of further expertence. The ICDA is nnsuilable for diagnostic classilica- 

m its present slate, but a modification developed over the yeats by the 

Royal College of General Practitioners has proven useful . 3 

The layout and terms used on the Howard encounter form are less 
ormtdab e than those on the Hopkins form and they may well be sufficient. I 
like the functional terms employed. Episodic Care, Initial Health Assessment, 
ong-Term Health Care Management and Routine and Preventive Care 
together with the No-Disease grouping that includes Family Planning seem 

Durrnionof ,1 116 H ° PkinS ,niXlUre 0f PUrp0SCS ’ P rocedures and treatments. 

Duration of the encounter is important, particularly if we are to look for places 

where time may be saved and productivity increased. There are arguments for 

ad hoc studies that keep track of each patient’s temporal movement through a 

dime by stamping the time that he starts and completes each phase of the 

medical care process, but I doubt that these are advisable for routine use 

The Harvard form makes a most useful contribution, in my view, in its 

classifies, .on scheme for assessing the Activity, Certainty, Severity and Rate of 

Progression of the patient’s problem. Their printout also refers to the 

Resolution of the patient s problem; this seems a desirable additional category 

liLTor dedSi ° nS c h3t CMniCianS ’ bC tHey Physicians ’ 

assis ants or nurses, must make when first encountering a patient and his 
problem How active is the process? How certain is the clinician of the 
diagnosis? How severe is the problem? How rapidly is it progressing or 
resolving. I would add a sub-category under management caJled Need for ' 
Intervention e.g., Emergent, Urgent, Elective or Expectant. Disposition is an 
important phase of the medical care process, and it should certainly be 

—rrr P 7 VideS eVidenCC ° f P rofessional a " d organizational 
responsibility and of continuity ,n patient management. We included this 
category on the Hopkins form. 

I see no reason why it should not be possible to at least standardize the 
terms used on encounter forms by the several university medical care plans. If 
all, or several of the plans, would adopt a number of the same categories,' 
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classifications, and definitions, it might be possible to achieve some real 
uniformity and enhance the prospect of comparisons. Unless such steps are 
taken, comparisons will not be possible. 

Patient management requires clinical data of the type described on these 
encounter forms and the variations suggested above. Institutional and 
organizational management of resources and facilities to meet patients’ needs 
requires that the clinical information be related to personal data and to 
financial data. Insofar as personal data are concerned, the major problem of 
patient identification needs to be addressed. There is a need to develop 
uniform systems of patient identification. This is widely recognized in 
relationship to hospital care, and a sub-committee of the National Committee 
on Vital and Health Statistics is now considering methods of standardizing 
patient numbering systems. 4 The United Kingdom is seriously considering the 
introduction of record linkage for all hospital discharge data nationally. 5 There 
is every reason to think that similar advances will be made in the United States. 
The university medical care plans should certainly be in the forefront of this 
movement. 

If the use of ambulatory care is to be related to in-patient hospital care, 
linkage between the ambulatory phases and the hospital phases of care should 
be achieved by some form of positive identification. The advantages of 
developing a system that would be compatible between the several university 
medical care plans is persuasive. There is first the opportunity for comparisons 
among the plans. In addition, patients enrolled in one plan when they move to 
another metropolitan area where a similar plan exists are, assuming that the 
plans are successful, more likely than other new immigrants to transfer to the 
local university plan. The possibilities over the years of following patients 
would be enhanced by a uniform identification system. There are three general 
numbering systems that are currently being considered. First is the Social 
Security number. The problem here is that the number is not assigned until an 
individual either opens a bank account or enters the labor market. There are 
also internal problems with the number itself and with its distribution 
nationally. Second is the birth registration number. It provides the best 
long-term prospect, but it will be thirty to fifty years before adults carry these 
numbers. The third system is the Hogben Identification System that utilizes 
the patient’s name at birth, date of birth, and perhaps one other form of 
positive identification. These are technical matters that need to be addressed 
by some group and the universities are in the best position to do this. Each 
system lias advantages and disadvantages. 

In connection with personal data, there is a need to express all tables 
describing the university medical care plans in compatible age groups. I suggest 
that the WHO age categories be used. These groupings break in the middle of 
the decades, i.e., under 1, l to 4, 5 to 9, etc. This arrangement avoids digit 
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preferences that are associated with inaccurate reporting of ages; Marital status 
data should be classified in identical fashion by, all plans. There is an even 
more important aspect of the patient!s household arrangements that merits 
attention. In relationshipjo^patfent care, it is frequently more important to 
know the patientYIiving arrangements that it is his martial status. Accordingly, 
classifications of living arrangements that include Living Aic^ne, Living with a 
Nuclear Family, Living with an Extended Family, or Living in an Institution, 
may prove useful. 

The third broad category of data relates to financial matters. The only 
useful contributions to this problem that I could find in the statementsYrom 
the six university medical care plans was that from the UCLA plan. The 
adoption of the accrual system of accounting in contrast to the cash 
accounting system seems an important advance. The former is the method used 
by contemporary industry. It permits the identification of costs for supplies, 
merchandise or sendees when they are received and used, rather than when the 
invoices are paid. This permits a closer examination of the relationships of 
revenues to expenses by each cost center or department. Basic to this method 
is the establishment of functional cost centers along the lines proposed by the 
American Hospital Association. 6 Some 100 classifications of expenses and 
revenues and 200 cost centers are advocated for the hospital environment. By 
clearly defining these cost centers and by associating them with defined 
elements in the medical care process that are in turn linked to patient problems 
and to the outcomes and end-results of care* there is a possibility that some 
real leverage may be possible on individual elements of the patient care systems 
and that meaningful comparisons can be made between plans. 

The use of relative value scales seems to be less satisfactory since this 
approach tends to be based on empirical justifications for charges rather than 
on actual assignments of identifiable costs. The concept of episode of illness as 
advance by Solon et al.. provides a temporal dimension that permits grouping 
processes in relationship to the patient’s presenting problem. 7 A good deal of 
work needs to be done in this area and the Howard classification scheme is a 
useful first step. 

One additional problem merits consideration. If the several university 
medical care plans are to be compared, it is important that the denominators 
be clearly defined by geography, by enrollment and by demographic 
characteristics. There is also a need to define the time dimension. It would be 
helpful if enrollment dates could be standardized among the plans or, at least, 
established on a semiannual basis so that there will be some measure of 
stability for each plan’s denominator. The possibility exists that person-months 
of “exposure” to care from the plan might be a reasonably satisfactory 
substitute. 
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To summarize these comments, I suggest that the accounting data system 
be developed in parallel with the clinical data system and the personal data 
system. The establishment of cost centers and the adoption of the accrual 
method seems desirable. Uniformity in the basic categories, classifications, 
terms and definitions employed by all university medical care plans should be 
encouraged. A classification scheme based on the major types of Episodes of 
Illness and Units of Long-Term Health Care and another based on categories of 
Patient Problems should be developed as soon as possible. All data should be 
expressed as rates with age and sex standardization to an agreed standard 
population. 

Now let me attempt to link processes to outcomes. If the objectives 
defined by the Howard and UCLA groups have any validity, it would seem 
important to maintain on the Patient Encounter Form, or on related records, a 
running account of levels of patient impairment. Some of the problems 
associated with these measurements are reviewed in the UCLA evaluation plan 
prepared by Houten and Du Bois. 8 The easiest way to obtain this information 
is to ask the patient at the time of his visit about the number of days in the 
previous week or two weeks that he has been unable to undertake his usual or 
customary activities, the number of days he has spent in bed, the number of 
days (usually in the past 48 hours) that he has experienced pain, and the 
number of drugs he has taken during that period. Expressed as standardized 
rates for the enrolled populations, these reflect the burden of impairment 
brought to physicians. 

The most difficult phase of data collection for outcome data occurs after 
the patient leaves the physician’s office. When disposition of the patient 
includes a return clinic visit, data about impairments may be collected on the 
second and subsequent visits. After the terminal visit for each episode of 
illness, however, the patient is not seen again. For some forms of episodic care 
that are self-limited or that are resolving satisfactorily, it may be difficult to 
ask the patient to return. A useful method is to employ a prepaid postcard on 
which the patient reports his status, including days of impairment during the 
intervening period since his last visit. Where postcards are not completed, a 
telephone call from the nurse or physician’s assistant is the only responsible 
way to deal with failure to ;omply; it is the same approach that should be used 
with a broken appointment. These arrangements permit measurements of 
patient care outcomes as reflected in impairment that can be related to 
processes and to patient problems. The periodic household survey is another 
method of developing cross-sectional population-based estimates of impair- 
ment, disability, discomfort, and drug utilization. Such measures are necessary 
for estimating the burden of impairment for those patients who live within the 
geographic boundaries of the plan, but who are not enrolled, and for those 
patients who are enrolled, but who do not use the available sources. 



What arc some of the process measurements that might be helpful in 
patient management and in medical care plan management? The possibilities 
are endless, but let me suggest a few: 

1 . Visits, laboratory tests, x-rays or prescriptions; 

2. Normal or abnormal laboratory or x-ray findings; 

3. Preventive care visits, immunizations, or health reviews and, 

4. Cost of all ambulatory, in-patient care, laboratory tests, x-rays or 
prescriptions, using the following parameters for each of the above 
items: 

-per thousand activity limitation, bed disability or discomfort days, 
- per thousand short-term episodes of illness (URIs, UTIs, Gl 
infections, etc.), 

-per thousand patient-months (or other units) of long-term health 
care management, 

-per thousand patient-days of hospital care, 

-per physician, nurstf, enrollment module, or health team; and where 
pertinent ne; thousand “no need to return” or “return pm” 
dispositions; or per one thousand night calls or emergency room visits. 

5. Activity limitation, bed disability or discomfort days 

-per problem type, e.g,, otitis media, UR1, essential hypertension, 
cardiac failure, peptic ulcer disease, etc. 

-per physician, nurse, enrollment module, or health team. 

6. Laboratory tests, x-rays or prescriptions, 

-per thousand visits by mean duration of patients 1 waiting times, 

-per thousand visits by medical care plan personnel absentee rates; 
-per thousand visits by non-professional labor turnover rates, 

-per thousand patient telephone calls by physician, nurse, enrollment 
module, or health team. 

All of these sets of data should include appropriate age and sex 
denominators, should be age and sex standardized for comparisons between 
physicians, health teams, or enrollment modules, and for comparisons between 
medical care plans. 

Perhaps enough has been presented to suggest the directions in which I 
think process evaluation should develop. Patient problems should be linked to 
processes and both to outcomes. Personal, clinical and financial data are the 
critical categories that need to be considered. Rather crude population 
measures are a first step. They will suggest other relationships for future 
examinations that may explain observed variations. A few interesting tables 
prepared for clinicians and for administrators are to be preferred to endless 
piles of computer printouts that are rarely examined by anyone and that have 
little to do with effective and efficient patient care. 
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DISCUSSION . 

ON EVALUATION OF MEDICAL CARE PROCESSES 
Chairman: Paul M, Densen 

The participants in the discussion agreed that in addition to their 
common purpose of attempting to serve patients in more effective and 
satisfactory ways, the university medical groups must also make a determined 
effort to measure the degree of success in achieving their objectives** 

A fundamental consideration in any evaluation of medical care process is 
a determination of the actual use of the plan by the population it is designed to 
serve. This approach should attempt to relate the services, manpower 
allocations, and expenditures to the requirements of the population it is to 
service, and subsequently to the results of the care which has been received. 
The discussions concerning specific procedures which might be used by the 
medical care plans focused on the design of a flexible record system, which 
would serve several purposes: to foster good patient care; to provide 
management with data for operational needs; and to provide a basis for 
research and for evaluation. 

RECORD SYSTEM 

The potential for investigating factors which may influence the patterns 
of use and the patterns of delivery of care lies in the design of the record 
system of the medical care plan. The participants faced this issue in their 
attempt to reach a clear concept of the target population to be served by the 
plans; and in their discussion of ways to obtain information which would 
permit comparisons among different plans. 

In trying to form some general guidelines for items to be included in a 
basic record system one major rule underlined by the participants waMhef 
importance of limiting information obtained on a routine basis to datFessential 
to the day-to-day operational management of the plan; another was the need 
for flexibility in the system which allows the collection of data not only for 
day-to-day management purposes but which provides the starting point for 
research into specific questions which may arise. 

The participants considered as critical the question of how to obtain the 
array of items discussed by Dr. White and urged that attention be directed to 
those items which should become part of a basic record system to be collected 
and tabulated on a routine basis, in contrast to those items to be collected but 
tabulated only on an ad hoc basis for special studies. These distinctions were 
considered essential to prevent overloading the record system to the point 
where an endless amount of routinely tabulated data interferes with the 
management of the program. Obviously the point at which one draws the line 
is to some degree a matter for the individual plan, and decisions referring to 
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recording, tabulating, and retrieving data have to be made at each step along 
the way. It is the price the plan has to pay for specific information that is the 
crucial factor in making these decisions; in other words, the decision depends 
upon what the plan may be required to forfeit in order to accommodate the 
data. A proposed general rule was to collect li what comes naturally. " 
Therefore, who is seen, by whom, and for what purpose were considered 
minimal requirements. 

Although a great deal of thought has been given by the medical plans to 
the mechanics of a record-keeping system, there had been limited consideration 
of the kinds of questions one should ask to obtain measures of how well the 
objectives of the plan are being met. One crucial point emphasized by the 
participants was the lack of a conceptual viewpoint based on epidemiological 
principles in the approach to the record system which limited the system to the 
mechanical function, rather than relating it to other aspects of health care. The. 
record system should serve as a link between health services provided, 
continuity of care offered, and quality of patient management. A well designed 
■ ystem is essential for assessing patient care and it should provide data for 
continued monitoring to ensure that the program is sound, the quality of care 
acceptable, and that problems requiring research or additional study are 
indicated. To make an assessment of the contribution of any of the university 
medical care plans to the total fabric of health services in the community, it is 
vital for each plan to know not only the extent to which it reaches the 
population it seeks to serve, but also the degree and manner in which its plan is 
used. Utilization data are required for two major purposes: 1) for managerial 
control which permits effective and efficient allocation of manpower, services, 
and facilities; and 2) to serve as a basis for planning and evaluation. To serve 
these purposes, the utilization data must be so organized as to permit 
comparative analysis both within the plan and between plans. To make such 
.analyses it is essential that comparisons be made in the form of rates which 
relate tire number of events (or services) occurring in a given period of time to 
the number of individuals who could experience the event (or risk of receiving 
service) during the same period of time. Three items are essential fora rate to 
have epidemiologic usefulness: the numerator (number of events of interest): 
the denominator (the population among whom the events of interest' are 
observed); and a specific period of time. In considering prepaid medical care 
plans the rate could be defined as: 

Number of physician services received by the population 

enro lled in the plan du ring the year 

Average number of persons enrolled in the plan 
during the year 

The participants noted that most of the plans have information about the 
numerator, but a major problem of such plans as the neighborhood health 
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center is the determination of the denominator-the size of the population 
among whom the events of interest are observed. 

The population enrolled in v a university prepaid plan is covered by a 
contract which terminates when an enrollee leaves the plan. . This provides the 
plan with a mechanism for maintaining a reasonably up-to-date enrollment file 
which makes it possible to obtain basic population data and to observe not 
only change in the number of enrollees from month to month, but also the 
comparative change from month to month- or from year to year. It is this 
comparative change which provides the basis for management control and sets 
the stage for program evaluation. 

The population in many neighborhood health centers, on the other hand, 
is quite different from that of the enrolled population in a prepaid plan, and it 
is much lore difficult to define or count accurately. This client population is 
usually generated by having the members of the family of a patient register to 
receive services at the center, or by attempting to register the people living in 
the neighborhood through some form of canvass. The registration process 
requires no contractual arrangement, and therefore, there is no systematic way 
of determining when a registrant leaves the area or stops using the center. 
Therefore, plans serving registrant populations have difficulty in determining 
true rates of utilization. For these reasons, rates based on registrants may not 
be comparable to rates based on enrollees because there are differenLselection 
mechanisms at work. 

The most common measure of the amount of service rendered by a plan 
is the total number of physician services or encounters provided during a given 
period of time, divided by the reference population (enrollees or registrants in 
the plan) during the same year. But it is also often important to determine the 
distribution around the average, as well as the average itself; that is, to be able/ 
to determine how many enrollees did not utilize the plan in a year, and how 
many have one, two, or three, or more encounters in a year. Knowledge of this 
distribution makes it possible to determine who are the high and low utilizers, 
what their characteristics are, and whether there is a relatively small proportion 
of persons in the plan who receive a large volume of the services. These data 
provide invaluable information for managerial control and epidemiological 
research. For example, two different systems of health care might have the 
same average number of encounters per 100 person units, but quite different 
distrib utions around that average. 

However, in order to determine the frequency distribution it is necessary 
to design the record system so that the unit of analysis is the individual. It was 
suggested that record systems be designed so that the frequency distribution 
could be calculated if desired, even though such a distribution is not required 
on a routine basis. This means that it is necessary to bring together all the 
encounters on a given person during a stated period of time. Without this 
ability to count the people who receive the services as well as the services 
thci elves, it is impossible to determine how effective the plan has been in 



reaching its target population, and it is also practically impossible to compare 
the experience of one program with another. 

In order to count people as well as services it is essential to have a way to 
link all of the services received by a given person. In a majority of the plans this 
link is obtained by assigning an identification number to each person enrolled 
m one of the group prepaid plans or to each person registered at a 
neighborhood center. Each service received by this individual should be 
identified with the person's identification number. Through the use of 
appropriate computer devices it is then possible to bring together all the 
services received by a person with a given number. 

In addition, if the plan is to get a picture of its utilization it is important 
to make sure that all the services it renders are counted. There may be some 
organized system to ensure completeness of reporting of services. Many plans 
have begun to adopt an encounter form to record the number and kinds of 
services rendered to the population which they serve. However, the use of 
encounters to measure services poses two major problems. First the recent 
development of allied health personnel who take over many of the duties that 
the physician performed in the past means that today the equivalent of what 
was once the average number of physican services per person per year has 
become the average number of encounters with primary providers per person 
per year. In this context, the primary provider is either a physician, dentist or 
other person who has responsibility for assessing the condition of the patient 
and exercising independent judgment as to care of the patient. Therefore, it is 
essential that the plan develop its internal structure in such a way that these 
encounter forms are filled out systematically and routinely each time an 
encounter occurs with a primary provider. 

The second problem relates to the plan’s need for the number of 
encounters with all staff members who provide a service. This information is 
needed for internal use by individual plans for looking at manpower utilization 
and costs. These data should be obtained on a sample, special study basis. For 
purposes of recording basic data, the encounter form must make it possible to 
indicate the nature of the encounter and the type of provider seen. 

The participants also noted that precise data on the type of provider are 
essential for those plans which receive per capita or other type of payment for 
Medicaid or Medicare eligibles. Since reimbursement under such arrangements 
is usually based only on physician encounters, the services of physicians must 
be clearly distinguished from encounters with other primary providers. 

The participants observed that the total utilization rate is of very little 
value either for day-to-day operations of the plar or for purposes of evaluation 
or research. It is knowledge of the major variables affecting the size of the 
utilization rate which is essential to the plan’s day-to-day operations. There are, 
of course, innumerable factors influencing utilization, and it is obviously 
impossible to take account of all those variables in studying the patterns of 
utilization. However, there are certain major variables such as age and sex 



104 



99 



TF 



which have great impact on utilization. It is essential that the administrator of 
a medical care plan have knowledge on a regular basis of the influence of these 
variables on the utilization of his plan. A basic principle suggested by the group 
is tha* for every characteristic of the plan’s population for which one wishes to 
examine utilization on a routine basis, it is necessary to obtain information on 
this variable both on the encounter form and on the enrollment or registration 
card. It was agreed that it would be desirable to develop an encounter form 
which as a minimum would provide the basic data required for routine analysis 
but which could be expanded as desired by individual plans in accordance with 
their special requirements. 

MEASURES OF PERFORMANCE 

Several different approaches to measuring performance were discussed. 
Among the standards of comparison suggested were the use of data from the 
National Health Survey or other national statistical series and the use of norms 
generally agreed upon by experts in the field. 

For instance, National Health Survey data indicate a utilization rate of 
about 4.5 physician services per person per year for the country as a whole. 
After appropriate adjustments for differences in demographic composition, a 
plan can determine whether its own utilization rate is generally high or low. 

Similarly it is generally agreed that it is desirable for prenatal care to 
begin early in pregnancy. A plan can measure its performance in this regard by 
examining the trend in die proportion of pregnant women who come in for 
prenatal care in the first trimester of pregnancy. In the same way, pediatricians 
recommend that certain immunization schedules be adopted in caring for the 
young child. A plan can use these recommendations -as-norms against which to 
judge its performance. Whenever possible such norms should be based upon 
scientific evidence which establishes their validity. 

The participants agreed that while the use of statistical series or norms is 
useful in giving the plan an indication of its performance relative to some 
average, thus providing a basis for further examination, it is difficult to tell 
whether any differences found between the plans performance and the 
standard of comparison are due to the special characteristics of the plan or to 
outside influences. To determine how much of any observed difference is due 
to the plan requires carefully controlled special studies. 

As an example, the point was made that many of the plans stated as a 
goal a reduction in the incidence of hospitalization. While the operating data of 
the plan should provide information on the rate of admission and length of 
stay in the hospital and these data can be compared with available published 
data, the determination of whether any observed difference is the result of the 
plan’s efforts requires a carefully designed study. The participants then went 
on to discuss potentials of this approach. 
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SPECIAL STUDIES 



Several of the medical centers outlined special studies and specific 
methods for gathering data which might be used to compare and/or assess the 
performance of their plans. These methods include: analysis of utilization of 
hospital services by a study and control group setup for this purpose by the 
Washington University plan; periodic surveys of the population, prior to and 
after enrollment, to be conducted by the Yale plan; analysis of before and after 
information on use of services based on survey data obtained by Johns Hopkins 
in both Columbia and East Baltimore; the UCLA plan which will attempt to 
compare patients served by different medical care teams. A crucial question 
raised by the discussants was, “How do you relate the changes in the population 
as determined by the survey or other data to the impact of a specific plan?” 

As previously indicated, the absence of an accumulation of health service 
data on a community or population basis makes it not only difficult to 
measure a plan's performance in meeting its objective of improving the health 
of its members, but also makes it difficult to relate changes to any plan’s 
medical care program. The series of studies being conducted in the area 
surrounding Harlem Hospital is an attempt to gather baseline and subsequent 
comparative information. Although a small on-going survey of several hundred 
families, sampled from a population of 400,000 people, is conducted monthly, 
the survey data for a two and one-half year period are not sufficient to point 
out factors which might be used as crude measures of the impact of the 
program on the population. 

Once again the issue of how to evaluate the impact of a specific plan on 
the reduction of hospital admissions was discussed. The difficulty in using 
hospital rates from Blue Cross or commercial insurance plans as a yardstick, 
and similar difficulties in using rates from Medical data, were reviewed. The 
problem cited was that the present record system is not geared to providing 
data on a population base. For instance, most Blue Cross plans can give a count 
of the number of contracts in force, but for purposes of measuring utilization 
the number of contracts is a poor unit for analysis. Until the conceptualization 
of the need to be able to count the total number of individuals in the plan at 
risk of being hospitalized becomes universal, the group concluded that for 
comparison of hospital rates and other rates relevant to patient care HIP and 
Kaiser Permanente will continue to be a basic source. However, for the kind of 
evaluation desired by the group, special studies will be necessary to obtain 
information on use of hospitals by persons in the university medical care plans. 
To study the impact of various forms of providing health care or methods of 
payment on the incidence of hospitalization, it is necessary to know the extent 
to which persons covered by the plan use hospitals. This information must be 
obtained either directly from the hospitals by special arrangement with the 
medical care plan or from a centralized file. However, at present such a system 
is the exception rather than the rule due to the difficulties already mentioned. 
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II there are numerous hospitals in the area, it is easier to use a centralized Hie 
than to make separate arrangements with each of the hospitals. Another 
advantage to a central source of information is the ability to estimate the 
extent to which persons go outside the plan for hospitalization even though it 
means out-of-pocket payment by the individual. 

In addition to looking at hospitalization, administrators are looking at 
other aspects of medical care processes within their plans to assess the 
effectiveness of the program. Special studies involving surveys or other 
methods may be required for examining such issues as the impact of the plan 
on severity of illnesses, disability days and impairment, continuity of care, 
patterns of referral, satisfaction, and the use of services outside the plan. 
Although most of the plans indicated some type of initial survey and a few 
discussed periodic sample surveys, the high cost of collecting, tabulating, and 
analyzing the data received much attention. 

SATISFACTION 

Participants spoke of the problems involved in measuring patient and 
provider satisfaction. Measures such as: waiting time, broken appointments, 
failure to follow through with referral or medication, and compliance with 
prescribed regimen were discussed; the hypotheses being that if the plan 
members are satisfied, there will be fewer broken appointments or better 
compliances. In essense, these issues could be used as some indices of 
satisfaction. However, as Mr. Axelrod noted, there was limited information in 
the preliminary descriptions of the university plans about how they intended 
to look at the issues of attitudes, behavior, compliance and satisfaction of 
patients. A major problem is the difficulty in conducting a detailed study of 
patient satisfaction on a routine basis. Although the record keeping system 
gives an indication of the efficiency of operation and offers some dues, such as 
appointment status, referrals made, number of persons who have not used the 
plan, special studies are required because the degree to which any of these 
indicators really reflect patient satisfaction is not known. 

The group discussed the possibility of using some form of direct 
questioning or survey mechanism. It was pointed out that numerous 
experiences in the use of satisfaction surveys with large groups, such as the 
California State Fmployees and the enrpllees in HIP, repeatedly show a high 
proportion of satisfaction among members, but these are broad surveys about 
the general satisfaction response, and they do not supply data about 
satisfaction in specific situations. It may be useful to approach the issue by 
looking at the components of care. In this way one has an opportunity to get 
at possible sources of discontent. Key issues explored by the group were; 1) 
how to interpret the percent of satisfaction, and 2) how data on satisfaction 
are to be used. Several important operational uses for such data were detailed 
in the following manner: I) to alert the system to the difficulties in which it 
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may become involved, 2) to alleviate problem possibilities, 3) to demonstrate 
need to link behavior of individual and measures of satisfaction on an 
epidemiological basis, 4) to provide data for management decisions such as 
increase in manpower, change in time schedule, or number of hours the facility 
is open. 

It was agreed that some form of direct questioning of members could be 
useful in learning their attitudes toward specific components of care and could 
provide critical data for use in operational decisions. In such plans as the 
Medical Care Group of Washington University, or in others where there is a 
control group, results of patient surveys can be related to ’(he characteristics 
of the respondents and to their specific kinds of medical care experience. In 
this way, physicians and administrators can learn what particular practices are 
liked or disliked by particular types of patients. 

The design of questions which will accurately reflect patients’ reactions 
to the conditions under which they receive their medical care is. however, 
particularly difficult. Patients often view the medical care process along three 
quite separate dimensions: its acceptability; its accessibility; and its availability 
♦ or to use another frame of reference: the cost and convenience of the process; 
the perceived quality of professional and technical activity; the humanity with 
which care is rendered. Patients also have different expectations and different 
criteria by which they judge “good” care. For example, people in some sub- 
cultures want privacy; for people in others, care under such conditions might 
be considered isolated and unfriendly; for some, care is believed inadequate if 
no medicine is given; for others, if no x-ray is taken. In addition, patients may 
be reluctant to state their true feelings in fear of reprisals should their answers 
oecome known. 

Many of the university groups indicated that channels are open to bring 
consumers’ 'views directly to the policy-making levels. Consumer representa- 
tives should work with the plans to develop criteria by which to measure 
feelings and attitudes about the medical care they receive and to review 
methods proposed and questionnares to be used by the plans in conducting 
necessary-studies. Not only is there a need to measure consumer satisfaction, 
but there are also needed methods to measure the administrative and profes- 
sional aspects of the plan which affect satisfaction among staff. The scales 
generally used to assess satisfaction are limited and need to be reviewed, re- 
fined, and tested. After criteria for measuring the components of satisfaction 
have been developed and validated, the plans will need to develop a comparable 
method for collecting specific information and applying these scales to their 
own data. 

NATIONAL IMPLICATIONS 

The lack of adequate health data accumulated over a long period of time 
makes the question of measuring performance of the plans quite difficult. 
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There are relatively few statistical series, other than mortality, in the health 
care system which can serve as indicators upon which to build valid evalua- 
tions. The National Health Survey lias been available for approximately fifteen 
years, which is actually a very short time span for producing anything except 
crude measures. \ 

However, as efforts are made on the national scene toward restructuring 
the health care system to make it more responsive to the needs of the people 
and toward changing the mechanisms for financing health services, some form 
of evaluation and accountability will be required. The general rules have been 
set forth. There is need, the participants concluded, to review the proposed 
plans to determine if there are opportunities to reflect the requirements of the 
population: 1) in various measures of utilization collected on a routine basis 
for purposes of management control, and 2) in special studies designed to 
assess the significance of the proposed programs within the fabric of the total 
health care system. 
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OPENING COMMENTS ON OUTCOME MEASURES 

Sam Shapiro 

Within the framework of this Conference, it would probably be most 
useful to view the subject of outcome as focusing on the measurable effect that 
a health care system, either in its entirety or in selected components, has on 
some aspect of health or well-being or on health care behavior generally 
believed to be associated with an end result. Under other circumstances, a 
discussion of outcome might include the impact of a health care system on 
utilization and cost but these elements are being taken up separately. 

Implicit in the issue of outcome is the need to make comparisons. This 
may tie directed at contrasts between the system and another setting or 
between alternative methods within the system or it may involve change in the 
system. The methodology may take the form of an experimental design or a 
comparison between groups with definable characteristics; observations may be 
longitudinal, cross-sectional, or of the before and after variety; sources of 
information may include health service records, special medical examinations, 
interviews (household or at the health care facility), mail and telephone 
surveys; the measures may be mortality, morbidity, impairment and dysfunc- 
tion in various manifestations, altitudinal; with the approach ranging from 
highly targeted, specific measures of a particular disease or condition to 
summary indices. 

The extent to which specific plans or ideas have been developed by the 
University based plans varies widely but a commitment to outcome studies is 
stated by each. I want to explore this interest from several standpoints. First, 
there is the matter of perspective. Necessary as outcome information is, a 
considerable amount of humility has to be exercised in laying out a program. 
The studies that have been conducted or are in progress are, for the most part, 
well known to you-perinatal mortality, mortality among aged Welfare 
recipients, evaluation ' f automated mul tiphasic screening, effect of periodic 
breasl cancer screening on mortality and two or three other projects. The list is 
not long, in part because of the relatively low degree of commitment to 
outcome studies until recent years, and in part because of the difficulties in 
structuring sound research in this area. These difficulties will continue and will 
discourage efforts to measure outcome unless along view is taken. 

Special, well-designed studies can and should be initiated; they will have 
a life cycle dictated by their own requirements. But, my reference to “long 
view” is related to another Kind of consideration. This is the desirability of 
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laying down the basis for routinely recording and systematically retrying data 
relevant to outcome. Just as the utility\of patient encounter information for a 
particular plan is most directly felt by the plan's management, so outcome data 
of even a rudimentary nature can strongly influence decisions within a plan. 
Further, just as utilization rates arrayed for several of the plans broaden the 
value of the information by displaying differences and similarities and directing 
attention to responsible circumstances, so a series of comparable outcome data 
for a number of programs could suggest the presence of an “effect” that is 
common to the plans and differences that might be investigated. 

What are some of the measures of outcome that might become part of a 
recurring series? Considerations in answering this question follow: the item 
should either be clearly an end result or generally accepted as a requirement for 
a desirable end result; the item should be already accepted by providers of 
service as necessary record data or a convincing case be made for its entry; t he 
item should be easily defined and reasonably reliable; and the item should be 
readily retrievable from existing records in the plan or in another agency. A list 
of items that might be candidates for inclusion are: 

1. Maternal and infant care-stage of pregnancy when prenatal care 
starts; complications of pregnancy and their control; prematurity and 
perinatal mortality rates; postpartum care and family planning 
instruction and adherence. 

2. Children -immunization status: pneumonia; otitis media, complica- 
tions of communicable diseases. 

3. Adults-stage of disease when diagnosed (cancer, diabetes); complica- 
tions of respiratory diseases; rates of controlled and uncontrolled 
chronic diseases, including hypertension, ulcers, diabetes, congestive 
heart failure; allergic diseases and extent under control; mortality. 

Tliis enumeration is incomplete particularly for plans with heavy 
concentrations of selected segments of the population (indigent, aged) and 
there may well be differences of opinion regarding feasibility or the 
significance of particular observations. Also, the items cannot conveniently be 
summarized into indices. However, they do represent a start in describing 
movement consistent with the goals of medicine and can provide the platform 
for intensive studies directed at effects on functional status and disability. 

The approach described would have consequences on the scope ol 
population characteristics for which data have to be available. Age and sex. the/ 
usual items covered in the enrollment process, would have to be supplemented’ 
by some measure of r jcioeconnniic status and in many plans by ethnicity and 
color. 

Medical chart information would have to be organized in a far more 
systematic way than is true today, whether computers are used or not. But. 
this is really not a new thought. The work that Weeds has been doing in 
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devising a problem oriented record is paralleled by others concerned about the 
need to improve the quality of record information. The start of new programs 
provides an unusual opportunity to link these efforts to the retrieval of data 
for outcome measures. 

It would be a delusion to believe that a practical working record, no 
matter how complete it is, could ever do the whole job of assessing outcome. 
The record could not be relied upon to contain the details required to measure 
functional status, for example, even for those conditions under active medical 
care. Effects of conditions that do not come to medical attention at all or are 
under care outside the system, and attitudes towards and satisfaction with the 
delivery system would, of course, be lost if records were the sole source of 
information. / 
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CRITIQUE ON OUTCOME MEASURES 

Lester Breslow, M.D. 

At conferences on health and health care which are so popular these 
days, two items are commonly mentioned in juxtaposition: (1) infant 
mortality in the United States is substantially higher than in several other 
countries, and (2) the United States needs better health care, in particular 
improvement in health care delivery 7 . The juxtaposition of these two items 
usually appears intended to convey the notion that if we improve health care 
we will lower infant mortality. 

HEALTH CARE AND OUTCOMES 

There is some evidence that different arrangements for health care are 
associated with differences in infant death rate. For example, studies of 
experience under HIP several years ago showed that infants born with medical 
care under that plan had lower mortality than infants cared for under other 
medical care arrangements in New York City, Selectivity, however (families 
with greater health sensitivay might have favored the I IIP plan) conceivably 
could have accounted for some or all of the difference. Factors other than 
medical care clearly do influence the infant mortality rate. Thus in California 
in 1%7, there were 30.1 infant deaths per thousand live births among the 
blacks. 18.8 among whites, 15,8 among Chinese and 13.2 among Japanese. 
Such substantial differences in infant mortality among the racial groups in 
California do not appear explicable by differences in medical care among these 
various groups. What data there are suggest that education rather than medical 
care may be an important factor. 

The frequently presumed relationship between the overall health of a 
group, for example, as indicated by infant mortality, and the medical care of 
that group is subject to challenge. Evidence on the, point is difficult to find. 
There are data, of course, indicating the value of some specific aspects of 
medical care, such as immunization against certain diseases, the cytology test 
for cancer of the cervix, and antibiotic therapy of pneumococcal pneumonia. 
The claim, however, that one kind of arrangement for medical care is 
significantly better for general health than some other kind of arrangcniu:* is 
hard to justify, no matter how commonly it is reputed. 

Another presumed favorable outcome of certain arrangements foi health 
care delivery,' mentioned at this Conference as in most discussions of the 
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matter these days, is that hospital use is lower with certain arrangements for 
medical care than it is with other arrangements. It is often implied in this 
connection that it is desirable to reduce hospital use. In the limited sense that 
decreased hospital use generally means that health care overall is less costly, 
some advantage is gained. Put low hospital use good for the patients? Take 
the case of hospitalization for maternity care. During a recent period, in 
California the average hospital stay for maternity care was two to three days, in 
Massachusetts three to four days, and in London six days. Which is preferred? 
From the standpoint of saving on hospital care, California has the most 
favorable experience. Is saving on hospital care, however, the most important 
criterion in this matter? How the mothers and children fare under the varying 
patterns of maternity care may be a more significant criterion. 

The relationship between health care and its intended and actual 
outcome appears to be rather poorly delineated. It is obviously essential in the 
evaluation of health care to specify the objectives of the health care and the 
means of determining whether these objectives have been achieved. Improve- 
ment in, or maintenance of, the health of a defined population is only one of 
several objectives of health care. If serious about these several objectives, it is 
necessary that we endeavor to evaluate them all. It may be possible to combine 
the evaluation of several objectives, as suggested in one paper here, for 
example, combining outcome in the sense of end-result of medical care with 
certain processes of the medical care with which outcome is closely linked, 
such as patient satisfaction. 

VARIOUS OUTCOMES TO BE CONSIDERED 

In approaching the evaluation of health care, several possible objectives 
deserve attention. 

On perhaps the most primitive level, patient satisfaction with the care 
needs consideration. The attitude of people towards physicians and other 
elements of health care today has evolved from the earlier attitudes of people 
toward healers in general. Popujar expectations of care, whether ritual dances 
for the excoriation of demons, prescribing medicines, giving “shots”, employ- 
ing certain laboratory procedures, or spending time with the patient, form die 
basis f°r patient satisfaction. This is a matter of considerable consequence. The 
arrangements for and processes of health care must somehow respond to 
patient expectations and hopefully satisfy them, at the same time possibly 
seeking to change the expectations. Perhaps most important, the degree of 
patient satisfaction probably influences the way in which the person will 
subsequently use health care. A substantial part of the so-called “hard core 
resistance” to certain medical procedures may reflect adverse experiences with 
health care earlier in life. All of the helping professions, including the ones 
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devoted to health, are finding it necessary at the present time to devote greater 
attention to the matter of client satisfaction. 

A second type of outcome of health care, which can more readily be 
measured is the reduction of specific mortality and specific morbidity. For 
example, one can measure mortality from cancer of the uterine cervix, or the 
rate of discovering the disease in an early versus a late stage, under different 
patterns of health care. Neonatal mortality and many other indices can be used 
to determine whether one arrangement for health care yields a different 
outcome than another arrangement. It is also possible to measure the 
frequency of specific preventable diseases, such as rheumatic fever, measles, or 
middle-ear deafness, in population groups served by different patterns of health 
care. 

A third possible outcome of health care is health in the generic sense, 
that is, physical, mental, and social well-being. Several efforts are now under 
way to define this concept and develop a quantitative approach to it. One view 
is that health may be regarded as a spectrum, along which any individual can be 
placed according to certain indices of physical, mental, and social well-being. In 
one, paper at this Conference this type of index has been called a health 
impairment burden. 

Another way of getting at the outcome of health care, in the sense of a 
direct result on the people served, is to measure the development of specific 
health reserves. Among these may be mentioned immunization against 
particular communicable diseases, lower blood pressure rather than higher, 
lower blood sugar rather than higher, and level of respiratory function. The 
future of medical care may very well rest upon a major shift in emphasis from 
the current complaint-response approach to health care over to a health- 
maintenance approach which would be measured to a considerable extent 
through such indices of health reserves. 

One rather subtle and usually overlooked outcome of health care is 
community mor?.ie. For example, one outcome of the Salud Clinic in Tulare 
County, California, which drew local people into a health care service where no 
health service had existed before, was an apparent improvement ^community 
morale. The latter was evidenced by the establishment of a newspaper initiated 
by clinic personnel and involving otheis in the community, by the growth of a 
community social center, and other changes in the life of the community. 
Possibly the impact of the so-called “free clinics” will be far beyond the 
treatment of venereal disease, drug abuse, and unwanted pregnancies; it is 
possible that an even more important outcome may be the influence on the 
morale of the youth groups that are rallying to the “free clinics.” 

In consiuering the outcome of health care on the health of the 
population being served one cannot, of course, overlook the paradoxical effect 
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resulting from iatrogenic disease. As the power of medicine becomes greater, 
this aspect of the matter may well become more important. 

SOME COMMENTS ON METHOD 

The above reflections on outcome of health care occurred to me in 
reading (lie discussions of this topic in the papers for this Conference. In 
addition a few comments on method may be appropriate. 

A major point is that any consideration of outcomes should focus on the 
whole population for which the health care is intended. While this may seem 
elementary, it is the most often neglected element of the situation. One cannot 
depend upon the usual health service record system as an adequate basis for 
determining outcomes, because the usual health service system does not reach 
all of the people for whom it is presumably intended and therefoie no records 
of hcaUh care exist for some of the population in the typical system. 
Completion of the outcomes picture requires information about the entire 
population, both the segment which uses the health service system and that 
which, tor whatever reason, does not. 

Evaluation of health care often requires longitudinal studies. One usually 
wishes to know whether there has been any change in the situation and this, of 
course, requhes observations in at least two and preferably more points in 
time. 

Also in measuring outcomes of health care it is necessary to give 
attention to influences other than the arrangements for health care. For 
example, the number of hospital days usedby a population is affected not only 
by the physicians who prescribe the hospital care, but also by the number of 
bods that are available for use, and by the methods of payment for the hospital 
care as well as the methods of payment for the physicians. In measuring health 
on the basis of disability days, that is, time lost from work, one must take into 
account the fact that different groups have varying benefits for time off as a 
condition of their employment and these groups may use the benefits to 
varying extents at different times. As an extreme example, a study of postal 
workers' disability in recent times would have been confounded by their policy 
and action to obtain a raise in pay. 

Still another common difficulty in studying the outcome of particular 
arrangements for medical care, such as muitiphasic screening, is the problem of 
the “contaminated” control group. One may set out to compare the outcome 
in a population which is given a certain procedure with the outcome in a 
carefully selected controEgroup thought not to have the procedure, only to 
find that the procedure which is being evaluated is by no means complete in 
the study group, and, as time passes, it is more and more taken by the control 
group. The comparison thus becomes rather blotchy. 
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DISCUSSION ON MEASUREMENT OF OUTCOME 



Chairman: Paul M. Densen 



The evaluation of outcome -of a new system of delivery of care or of a 
new treatment for a specific ailment, for example— is today considered an 
integral component of the process eventuating in the outcome. The meaning of 
the word “outcome” would seem to be clear, but outcome as a concept has 
many facets which call for review and discussion, particularly in the context of 
this conference. Indeed, the word itself raised questions in the minds of some 
people, and other terms, such as “end results,” were suggested, but the partici- 
pants concluded that the term “ outcome” or “outcome measures” was accept- 



The definition of outcome that set the stage for discussion was “the 
measurable effect that a health care system, either in its entirety or in selected 
components, has on some aspect of health or well-being or on health care 
behavior generally believed to be associated with an end result.” Several prin- 
ciples follow from this definition as corollaries. 

As an initial step in the evaluation of outcome, a clear determination has 
to be made of why it is that something is being measured. Put another way, a 
determination is required of the purpose in mind for entering certain informa- 
tion in the medical record or for undertaking some special study. As examples- 
and these have become more or less standard examples—is it a goal of the Plan to 
reduce hospitalization? to reduce costs? to increase membership satisfaction 
with the Plan? Does the Plan wish to find out if its ministrations are in fact 
reducing mortality, morbidity, disability? The Plan may, of course, have mul- 
tiple objectives. 

The participants agreed that if the findings about outcome are to have 
any meaning and any reliability, some rules of organization and procedure need 
to be observed. In the main, these are much the same rules as would be 
followed in any scientific research endeavor. First, the form of the outcome 
has to be known; that is to say, the subject matter under scrutiny has to be 
such that it is appropriate to the problem whose resolution is being sought, 
albeit sometimes indirectly. At the other end of the methodology spectrum: 
the data must be valid in the sense that the influences or conclusions drawn 
from them will be warranted. 

Second, the effect must be measurable. Thus, if a putative reduction in 
hospitalization is being investigated, records should provide the number of 
admissions and the number of days in the hospital. If a reduction in morbidity 
is the hypothesis, measurement may prove more troublesome, for there inter- 
vene questions and doubts about reliable diagnosis, questions about the pres- 
ence and severity of illness, and other questions of the same nature. Never- 
theless, quantifiable information must be found and agreed upon, and 
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terminology delineating the type of data being retrieved must be as specific and 
clearly defined as possible. 

Third, evaluation of an outcome implies some kind of comparison, other- 
wise how can it really be known that anything has been achieved -that hos- 
pitalization or surgery has indeed been reduced or morbidity mitigated. One 
method of comparison is the before-and— after model. The Plan compares 
later data, obtained a suitable interval after some procedure lias been instituted 
and when results are available, with the situation before t he new procedure was 
begun. A common practice is to compare the outcome observations with data 
for a general population, federal, regional, state, or local, but this is hazardous 
since the probability that like is being compared with like may be small. Better 
comparison is made with another system serving the same kind of population 
and providing similar services but with such marked differences as a different 
method of payment (prepayment versus fee-for-service) or organization (group 
practice versus individual practice). To put it somewhat technically, the two 
populations should be alike for all variables, but the one under test or scrutiny. 

The best comparison is made where a population or membership can be 
divided into two (or more) groups by some random process, one group consti- 
tuting the treatment group, the other the so-called control group. This method 
still raises a few questions, however. One is the moral question of holding back 
beneficial service, home care, for example from the control group. Another is 
the well-known phenomenon that, over time, control of the control group 
deteriorates as its members obtain the services being tested, grow weary of 
cooperating, or move away. 

Population has now been mentioned several times. If the outcome of a 
service is to be measured and evaluated, it should not be tested simply in terms 
of the patients receiving it; for true measurement of its effectiveness-and 
because health care systems are more and more assuming responsibility for the 
health of a defined population-the service needs to be considered in terms of 
the population from which the patients are drawn, usually the membership of a 
Plan. Counts of visits tell little about utilization of physicians’ services-whether 
there have been too many visits perhaps (overutilization) or too few (under- 
utilization)^ to make comparisons which lead to judgments, utilization rates are 
needed. Rates require a numerator, the number of events (visits), and a denom- 
inator, the number of persons among whom the events could occur— or who 
were exposed to the “risk” of having these events happen to them. 

There are times, however, when outcomes have to be analyzed without a 
population base or denominator. The results of some efforts, like providing 
transportation or a playroom for children, can still produce a noticeable and 
measurable increase in utilization, which fact can have useful administrative 
applications. 

If two or more populations are being compared, they need to be as alike 
as possible in their composition and characteristics. But even within the same 
Plan, there may be more than one class of membership (Medicaid and an 
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employed population); for reasons of reliability, it may be advisable to treat 
the Plan as, in effect, two Plans in the evaluation of outcome. 

Outcomes as they are related to a population cannot be analyzed apart 
from other elements of health and life. By and large, they should be evaluated 
against the background of environment, community custom and morale (in the 
sense of the lift produced among people by the establishment of a neighbor- 
hood health center), such factors as nutritional status, and so on. 

The participants cautioned that one should not be misled by such easy 
assumptions as that patients are following their physicians’ orders. In other 
words, one should be as certain as he can that the outcome and the procedures 
to which it is attributed have a reasonable relation to each other. Outcomes are 
also affected by the way the services in question are offered. And one needs to 
ask whether an outcome such as the reduction of hospitalization is indeed a 
favorable outcome. By the same token, it may be unwise to claim too much if 
outcomes are favorable. More time might be devoted to the study of the effects 
of factors such as those cited in this paragraph. 

Data for the measurement of outcomes are usually obtained in one of 
two ways: from routine records, such as medical records, or special studies. 
Routine information that will permit the measurement and evaluation of out- 
comes must be planned for, in terms of both its inclusion and its analysis. 
Many such items, probably the more reliable ones, have already been listed in 
the critique presented by Mr. Shapiro. The criteria for their selection are also 
enumerated. Thought might be given by some agency or collection of agencies 
to the development of a basic data set along these lines. 

Other items which have been put forth for inclusion, such as days of 
disability and days of illness, are flawed by their subjectivity; they are difficult 
to count meaningfully, particularly in low-income populations where few may 
enjoy sickness benefits and must, in a sense, disregard aches and pains. Once 
again, the population-including at least its age and sex composition-must be 
known so that rates can be calculated and used for purposes of internal admin- 
istration, for planning and for comparison. 

Despite the best of intentions, routinely collected data do not always 
supply all the desired information for measuring outcome (nor should they be 
expected to). This is particularly true in those milieus where innovation is 
stressed: where new procedures (administration and treatment) are tried, where 
research is emphasized. Not all Plans may be equipped to undertake the special 
studies they would like to, because of their small size (many phenomena re- 
quire sizable populations if significant differences are to be detected), lack of 
appropriate research-oriented personnel, even lack of physical space. But for 
those organizations with the capacity and will for research, grants from various 
sources are available to encourage and support their efforts. As was already 
indicated, if a Plan does embark on a special study, careful attention has to be 
paid to the design of the study, its operational procedures, and the precise 
nature of the information to be gathered. 
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Finally, the participants concluded that studies-routine or special-should 
not be confined to that subject matter which investigators know how to meas- 
ure. Granted that it will never be possible to measure every factor in a health 
care system, attention needs to be given to developing new measures in new 
areas. 
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COST ANALYSES AND DATA NEEDED 
FOR A BETTER APPRECIATION OF CHOICES 
IN THE PROVISION OF MEDICAL CARE 



Paul J. Feldstein, Ph.D. 



In order to specify the type of cost data that should be collected within 
each of the plans, it is first necessary to specify the purposes of having such 
cost information. This is an essential first step. If the purposes were not 
specified, then when it comes time to analyze the data for such purposes the 
relevant data will not be available. Similarly, just to collect cost data on 
everything at every point in time is very expensive and presumably we are 
subject to some budget constraints (although based on previous data collection 
efforts in the health field it may not appear so). 

Before turning to the type of cost analyses to be undertaken, I would 
like to make a few comments on the general subject of “costs.” Basically, why 
are “costs” important? In most of the plans presented, there is little or no 
attention paid to this subject. In some cases it is ignored while in others ii is 
handled in such a manner as to be best left to accountants and other 
technicians while the managers and conceptualizes of the plans can be left to 
work on the really important problems. 

If this is the case, then the accountants will develop that cost 
information which best serves their own purposes, e.g., a balance sheet and 
“profit and loss” statement, rather than prepare the type of cost information 
that will serve the needs of the managers. 

The apparent low priority given to cost analysis is, in my opinion, 
distressing since the subject of medical care costs have been and are currently 
of major importance in the provision and financing of medical care today. 
Unless proper attention is given at the beginning of these plans to the type of 
cost analyses that should be done, I believe the knowledge derived from such 
plans will have less of an impact than is hoped for. 

Costs are important because, together with other factors, they help us to 
make choices! Are new forms of organization worth it? Is the marginal value 
from certain changes in benefits, higher quality, etc., greater than the marginal 
cost of such changes? Unless the right questions are asked and the relevant cost 
information available to answer them, oui choices may not be as good as they 
could be and unfortunately, potentially good ideas may be incorrectly 
evaluated. 



\ 
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Although this paper is primarily concerned with explicit costs to the 
University Medical Plans, there are other costs that are involved in the delivery 
of medical care of which the managers should be aware in making choices. 
Minimizing the costs of. producing medical care in an institutional setting may 
shift some “costs” to the patients, their family and visitors. Examples of such 
costs are travel and inconvenience. Although it is more difficult to quantify 
these latter costs, they should be considered in determining the type of 
organizational frame work for delivering medical care. By knowing the explicit 
costs of different systems, it will be possible to make better choices with regard 
to these other implicit costs. 



THE MEASURE OF OUTPUT 

In order to undertake cost analyses, it is necessary to be able to relate 
costs to some measure of output or outcome. This is perhaps one of the most 
difficult aspects of cost analysis in the field of medical care. All of the plans 
discussed have multiple outputs or objectives. They may believe that it would 
not be appropriate to make cost comparisons between the plans. However, as 
soon as some of this data becomes available it is inevitable that comparisons 
will be made, and the persons making those comparisons may tend to neglect 
differences in the plans’ objectives because the costs were not separated by 
each objective or output. 

It is, therefore, in the interest of each of the plans to be able to specify, 
in a manner that would enable data collection, each of their output!,. Further, 
once such outputs have been enumerated, all costs should be allocated on the 
basis of each of these outputs. The measures of output. are important because 
they suggest the bases for allocating costs. /' 

Output, for purposes -of inter-plan comparisons, is defined to be a 
treatment for a particular diagnostic category, of a specific level of quality 
(which might perhaps empirically be defined as the probability of a successful 
outcome). 

Costs per treatment between organizations, therefore, must consider 
differences in the treatment (or diagnostic) mix as well as differences in the 
level of quality. Other factors affecting cost differences will be discussed, but 
in defining the cost per unit of output to be measured, it is probably necessary 
to be able to cost out differences in the level of amenities, to treat different 
diagnostic categoriei separately, and to adjust for levels of quality. 

If the above data were available, it would then be possible, presumably, 
to examine the relationship between the additional cost of higher levels of 
quality and the marginal benefits of moving to these higher levels. (The 
relationship would presumably be nonlinear.) 
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Since the measure of output is so important, because it suggests t he type 
of cost and medical data to be collected, I would like to make one further 
comment in this area. Although we are ultimately interested in “health,” the 
provision of medical care is only one factor influencing health levels. 
Therefore, if health levels are considered to be the output of a plan and 
comparisons are to be made between plans on these grounds, then data on the 
other inputs affecting health, e.g., housing, nutrition, personal health habits, 
etc., would also have to be collected and introduced into the analysis. 
Although worthy, this is a much more difficult task, and, based on the material 
received does not appear to be the output measure indicated by the plans. 

THE USES OF COST ANALYSIS 

There are several types c r cost analyses, differing in their purposes as well 
as the type of data needed to conduct them. For both internal (i.e. whether 
costs are minimized) and external decision-making (i.e. comparisons of the 
performance between several organizations delivering medical care), the 
following kinds of cost analyses should be undertaken: the extent of 
economies of scale for long-run minimization; optimum input mix for 
minimizing short-run costs. 

If data were available to conduct these types of analyses, then we would 
have the necessary knowledge to do the following: 

1. Set up a Cost Control System by diagnosis, by department, by 
physician and by institutional setting; 

2. Be able to forecast future expenses (by provider, diagnostic category, 
and physician) given changes in the patient or diagnostic mix, changes 
in prices and wages, etc.; 

3. Determine that mix of resources that is least costly for providing 
treatments and how the optimum mix of resources will change if 
there are changes in their wages or prices; 

4. Determine the scale of operation (for the entire treatment as well as 
for the individual ^ttings) that is least costly, i.e. economies of scale. 
Hence the ability to determine “make or buy” decisions in all phases 
of the operation; 

5. Have the information necessary for setting prices for the care received 
and also for determining premium levels on a capitation basis; 

6. Determine the causes of variations in costs between different plans or 
organizational forms of delivering medical care; 

7. Determine the “profit and loss” at different levels of operations, with 
different types of patient mix, and how profit and loss would change, 
given changes in any of the underlying factors affecting costs; 




124 



119 



8. Provide part of the necessary information required to conduct 
cost/benefit analyses of specific programs offered by the plans; 

9. Serve as a basis for incentive reimbursement experiments. 

This would have to be a much longer paper for me to describe in detail 
the importance of each of the above uses and how specifically each is to be 
achieved. Therefore, I would like to discuss for purposes of an example, only 
one of the above, the importance of knowing the extent of economies of scale . 

ECONOMIES OF SCALE 

The economist is interested in knowing the extent of economies of scale 
in an operation, i.e. the relationship between cost per unit and size of 
operation. Presumably a plan with a small volume (e.g. 500 families) may have 
very high costs per family per year, not because it is inefficient or producing 
high quality care when compared to another plan, but because there are certain 
economies in providing medical care in a comprehensive group practice setting 
that are achieved when the plan size enrollment reaches e.g. 30,000 families per 
year. For example, it is important to know what the relationship is between 
cost per family (or person) per year and plan enrollment. If there is a sharp 
drop in per unit costs with increase enrollment, then this size factor must be 
considered in making comparisons between costs of different plans. Further, if 
price is related to cost, then the ability to operate a larger scale may determine 
the financial success of a plan, e.g. per unit costs may be so high in a small plan 
as to discourage additional enrollment, whereas if it were known that per unit 
costs would fall a certain amount with increased size, then the additional 
enrollment could be sold at this future cost. 

Related to the overall problem of economies of scale for the entire 
operation of the plan, which as described above would be necessary knowledge 
for establishing (and predicting) a premium for enrollment in the plan, is the. 
use of the same idea within the plan for the operation of various services and 
operations. 

The extent of economies of scale are not the same for each service or 
facility. A plan of a given size may experience large economies of scale in the 
use of certain facilities or operations while at the same time be subject to either 
diseconomies of scale (too large) or have high unit costs in other operations 
because of the nature of the cost-size relationships in those other facilities or 
operations. It is important to be able to have cost/size data on all aspects of the 
plan’s operation so as to know where possible savings may accrue. For 
example, a plan with a small enrollment may determine that in certain 
operations it could purchase these services from other organizations more, 
cheaply than by providing it themselves. 



Therefore each plan should collect data on the cost-size relationship of 
all its activities, as well as its overall operation so that this information could be 
used for: 

• estimating the premium at different enrollment sizes; 

• determining which facilities and services it should operate itself and 
when it is cheaper to buy such services from others (and similarly, 
when it is less costly to provide it themselves or the savings from doing 
it cooperatively with others); 

• making cost comparisons with other plans (or methods of providing the 
same care) by adjusting for cost differences that are associated with 
size differences in order to establish cost differences due to other 
factors. 

A further example of the kinds of other data that would be required (in 
order to hold their effect constant) in addition to data on economies of scale, 
e.g. volume, when making an inter-plan comparison of costs, would be: 
product mix of the plan, e.g. diagnoses; quality levels: population served, e.g. 
age, sex; and wage and price differences of resources. 

Then, assuming an accurate allocation of costs (e.g. various activities such 
as training and research are not being subsidized) and adjusting for economies 
of scale, the differences between costs per treatment by diagnosis can be 
attributed to differences in efficiency of operation. * 

TYPE OF DATA TO BE COLLECTED 

111 order to be able to conduct the above cost analyses, it is necessary to 
have certain types of both cost and non-cost data. The’ following framework 
indicates the ideal type of data to be collected. 

Data for Use in Estimating Production Functions: A production function 
in economics is the physical relation between inputs and outputs. The output 
may be at various levels of aggregation, i.e. treatment of a diagnosis in a 
hospital setting to the treatment of a diagnosis ir all settings-outpatient, 
inpatient, home care, etc. For each diagnostic category there are various inputs 
and combination of inputs that go into providing a treatment (at various levels 
of quality). Therefore, the first step in determining what is the economically 
most efficient method of treatment is to have data on the inputs (personnel by 
skill level and amount of each category of personnel, supplies, and capital) and 
the extent to which these inputs may be substituted for each other in providing 
a treatment for a diagnostic category (again of a specified level of quality). 

Presumably the resources (labor and non-labor) used in providing 
treatments between each plan will not be the same, nor will they necessarily be 
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the same within one plan over time. By collecting data on those different 
combinations of resources, and through various statistical techniques holding 
constant other factors, it should be possible to determine empirically the 
degree of substitutability of these resources for each other. Knowledge 
regarding the extent of substitutability provides important information, if we 
are to be able to select tile least cost method of providing a given treatment. 

Data for Use in Estimating Cost Functions: Given the above data, it is 
then necessary to have data on the prices of the inputs, e.g., wages of different 
types of personnel, prices of supplies, and capital cost. This data then, prices of 
inputs and amounts and types of inputs by diagnostic category, will make it 
possible to determine what is the least expensive manner for producing a given 
treatment, of given quality. 

Once we have the above data, prices and amounts of resources, then we 
also have estimates of costs for producing treatments by diagnostic category. 
For example, if costs are related to all the outputs produced, e.g., x-rays, then 
it will be possible through use of the patient encounter form to determine the 
inputs that went into providing each treatment. The costs for each patient or 
episode of illness are then the costs of all the inputs indicated. We could then 
make inter-plan comparisons of the cost of treatment for a particular type of 
patient in a certain diagnostic category and determine the cause of such 
differences. Variations in costs might be a result of difference in the prices paid 
for the resources (wages differ by region) or of differences in amount of 
resources used. 

Such specific data then, on prices, resources, and outputs can be used not 
only to make inter-plan comparisons, and suggest the reason for cost 
differences, but can also provide information regarding the extent of 
economies of scale, as well as being used for cost control purposes. For 
example, with the above information it should be possible to estimate the costs 
for producing treatments by diagnostic category (further classified by type of 
partient)-hence its usefulness also for budgeting future costs. If for a certain 
patient or for a certain time period the costs of a treatment are higher than is 
expected, it should be possible to determine the cause of the sudden increase. 
It might be a result of an increase in use of certain resources or possibly that 
there was a wage increase that month; therefore not necessarily a cause for 
concern that there is inefficiency in the operation. 

The above data, i.e. types and amounts of each input, their prices and 
wages, the type and volume of output, are admittedly ideal. Many organiza- 
tions may have difficulty in providing such information. As a beginning, 
however, the plans should attempt to allocate all costs to each output. The 
above refinements would then be a phase to work toward. Even though less 
\ than ideal, the ability to relate costs to output would enable a number of the 
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above analyses to be undertaken. For example, it would permit, through the 
use of statistical techniques the separation of fixed and variable costs. 

Data for Use in Estimating Future Demands for Care: Another set of 
necessary data, which are not cost data, but are relevant for determining future 
costs, are estimates of the demand for care. Unless there is some idea of the 
type and quantity of treatments that will be demanded (also of different 
quality levels), it will not be possible to accurately budget costs (resources) for 
serving those demands. There may be too many resources, which would be 
wasteful or too few -which also would be costly, e.g. the patients’ outcome 
be affected in that they may have to have less care than if there were better 
planning. 

To summarize the kinds of data to be collected, it is necessary to have 
data on prices for each of the resources used, the type and amount of resources 
used, categorized by treatments for each diagnostic category. Further, patient 
information should also be available for the above, e.g. age, sex, severity of 
illness, etc. Tire above data should be available aggregated for the entire health 
plan (i.e., episode of illness in all facilities) as well as within each of the 
institutional or facility settings in which care is provided, e.g. outpatient, 
hospital, physicians’ office, nursing home, and even in the patients’ home. 

The above information will then enable comparisons to be made not only 
between different plans but for the same plan over time; also it should be 
possible to determine the extent of economies of scale in the entire operation 
as well as for each institutional setting; an estimate of efficiency can also be 
made— if quality and output can be sufficiently well measured; it should be 
further possible to institute cost control and budgeting procedures, and also to 
determine the premium for the plan’s services based upon expected demands 
for service. 

I have not been very specific as to exact types of data to be collected 
since one can get bogged down in estimating fringe benefits as part of wages, 
etc. But I have suggested that the data be collected in such a manner as to be 
able to relate it to treatments by diagnostic categories. This means price and 
quantity data on both human and nonhuman resources. Some resources do not 
vary (within short periods of time) regardless of the volume of treatments 
provided by diagnostic category, such as certain capital costs, while some 
resources do vary with the number of treatments provided. Both these “fixed” 
and.“variable” resources are to be included. Further, although the emphasis has 
been on medical care output, traditional methods of allocating data to 
departments or cost centers would also be useful, for purposes of cost control 
and budgeting on those centers. But in order for costs to be useful for the 
other purposes described above, it is necessary to be able to relate the costs to 
patients of different diagnostic categories. With the present state of informa- 
tion systems and computers as well as patient identification numbers, this 
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should be less of a problem in the future than it has been in the past, at least in 
those cases when it was not done for technological reasons. 

ADDITIONAL FACTORS INFLUENCING INTER-PLAN COSTS 

In the above discussion of the purposes of the cost analyses and the type 
of data to be collected, several factors affecting costs were omitted which must 
be considered if comparisons between plans are to be made. 

Payment Mechanisms: Each of the plans involved (or future plans) may 
differ in the method used to reimburse physicians (e.g salary, fee for service, 
capitation) or other providers of care (e.g. cost plus for hospitals). Differences 
in these payment mechanisms between plans may result in wide differences in 
the cost performance between plans because the incentives in how many 
resources are used in treatment and how well they are used may differ. In fact, 
if different payment mechanisms are not considered, then perhaps one of the 
most crucial factors influencing performance-incentives-may be omitted, and 
their effect attributed to other factors which would be misleading in their 
implications for future policies derived from these findings. 

In addition to possibly differing incentives to the providers of care, the 
incentives to the demanders of care may also differ (e.g. extent of coinsurance 
and deductibles, benefit coverage, etc.), hence also possibly resulting in levels 
of utilization and utilization patterns— hence, costs— that differ between plans. 

Differences in Plan Output: The plans involved (as well as future plans) 
differ with respect to other objectives, such as providing training, conducting 
research, and serving as the prepayment agency for the plan so that in order to 
conduct meaningful cost comparisons between plans and also for determining 
the costs for providing medical care it is necessary to separate out the costs of 
these other functions, or outputs. 

If adequately sorted out, then similar analyses as described above can be 
conducted with regard to each of these other outputs. 

Experience: If the various University Medical Plans believe that they will 
now be able to reduce medical care costs to their enrollees by about one-third 
because they now offer prepaid group practice, they will be in for a surprise. In 
addition to the higher costs that will be involved because teaching hospitals are 
being used, the higher premiums necessitated by being less than optimal size, 
the higher costs because it will be difficult to separate out their other outputs, 
e.g. research and training, there will be higher costs that are a result of being 
novices in operating a medical care plan. As experience is gained in delivering 
medical care, these latter costs will presumably decrease. Whether the other 
costs will similarly decrease will be determined by subsequent evaluation 
studies. 
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DISCUSSION ON COST ANALYSES AND DATA NEEDED 
FOR A BETTER APPRECIATION OF CHOICES IN 
THE PROVISION OF MEDICAL CARE 

Chairman: Sidney S. Lee 

There was general agreement among the participants on the need for 
more “cost information” in order to evaluate more adequately the various 
existing university medical care plans and to aid in the development and man- 
agement of new ones. 

However, a precise determination of the cost information to be collected 
in a health care program often proves to be an exercise in frustration. These 
attempts which are based on traditional methods of categorizing costs per unit 
of service fail to recognize the fundamental nature of the group practice pre- 
paid concept; and any effort to apply classic economic theory presupposes a 
level of sophistication and discipline which simply does not exist in the health 
care field. 

The participants suggested that the subject could perhaps be most mean- 
ingfully approached by considering the three primary prepaid group practice 
program needs, and then the kinds of cost data each requires. The three pri- 
mary program needs are: 

1. Planning and Development (i.e., determination of the operating and 
capital dollar requirements for a new program) 

2. Management (i.e., prospective development of a program’s financial 
requirements, on-going monitoring of performance, and allocation of 
alternative resources) 

3. Evaluation (i.e., internal analysis of cost centers and external analysis, 
or inter-plan comparisons) 

Before examining each of these areas in detail, the participants reviewed 
some of the basic underlying assumptions concerning the group practice pre- 
payment concept, and the elements that seemed to contribute to its success. 

The experience of medical care programs that have assumed responsi- 
bility for the primary health care of a population on a prepaid basis has 
consistently demonstrated the need for a number of elements to be operating 
in combination if such programs are to be, and remain, viable. Taken together, 
these elements contribute toward relating the financing and delivery of medical 
care in such a way as to relieve constraints against, and indeed provide an 
incentive towards, the provision of care in the most appropriate and efficient 
manner. These major elements include: benefit structure sufficiently compre- 
hensive to provide a wide range of services, locations in which services may be 
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rendered, and a variety of professional and paraprofessional staff to render 
these services; a financing structure sufficiently predictible and unrestricted to 
permit the development of facility and manpower resources for a defined 
population in such a way that alternatives to the use of the scarce (and costly) 
hospital and physician are encouraged; an organized group of health care pro- 
viders who share patient responsibility; and management skills-medical and 
fiscal-to assure that these elements operate in the proper balance. 

1. PLANNING AND DEVELOPMENT 

The critical issues in the planning and development of a new Plan, from a 
cost standpoint, relate to the determination of operating cost requirements and 
capitalization requirements. Discussion followed on these points. 

Operating Cost Requirements 

As suggested by the above principles, operating costs in a group practice 
program are most appropriately viewed per unit of population (as opposed, for 
example, to per unit service). This is so because of a ‘basic objective in the 
prepaid health system, i.e., the use of the most appropriate alternative resource 
in maintaining health and treating illness in a population. 

Cost per person is generally defined as the amount of money required to 
provide a given range of health services to an enrolled individual for a specified 
unit of time, usually per month. Under typical group practice prepaid arrange- 
ments, a subscriber (or a third party on behalf of a subscriber) pays such 
periodic, predetermined amounts to cover the cost of all, or most, of the 
services offered by the Plan. The Plan then assumes responsibility for using 
these monies to organize staff and facilities in order to meet most effectively 
the needs of its subscribers. 

Perhaps the most significant advantage of capitation reimbursements to 
the Plan is the flexibility of such reimbursement arrangements in terms of what 
services can be provided, where, and by whom, in reaching toward the objec- 
tive of providing care in the most appropriate fashion for a particular situation. 
This is in contrast to the more customary approaches of reimbursement for 
particular conditions, where care is provided in traditional settings by tradi- 
tional manpower. The implications of this increased flexibility in delivering 
comprehensive health services are exceedingly important. 

Prepaid plans have an advantage over OEO Neighborhood Health Centers, 
for example, where there usually is a lack of a denominator around which to 
build capitation reimbursement arrangements. 

Linked with a comprehensive benefit structure and the availability of 
physical and manpower resources, a capitation reimbursement arrangement 
provides a powerful lever toward an efficient use of the elements of the health 
care system. 
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Developing the Capitation 

The mechanical process of arriving at a capitation is reasonably straight- 
forward and probably consistent, with some variations, from one plan to 
another. 

First it is essential to determine the level of membership to be served. 
This might be based upon such factors as size of potential market (and ex- 
pected penetration of that market), availability of resources, and the optional 
size of an ambulatory facility. Most of the university medical care plans seem 
to cgree that the optimal size should be in the range of 30-50,000. It is within 
this range that the most efficient use of staff and costly equipment (e.g., x-ray) 
can be achieved. 

A second item is to develop a set of planning assumptions that set forth a 
given set of benefits and the utilization expectations (hospital days/1, 000/year, 
physician services/person/year, lab and x-ray services/person/year, etc.), and 
staffing ratios (physicians, nurses, paramedical, etc.), required to deliver these 
benefits. There is a wealth of data available, from Group Health Association of 
America and specific group plans, regarding the utilization of specific services 
and the manpower required to serve a prepaid membership. These data tend to 
be rather consistent from plan to plan and provide a valuable resource for new 
programs in their developmental stages. Such other factors as productivity and 
scheduling must be considered if the overall assumptions are to be meaningful. 

The planning assumptions must be converted into a budget. Assuming 
that the program does not own its hospital beds but pays for them as used, the 
hospital portion of the budget is merely the product of the projected hospital 
utilization rate and the estimated per diem cost for the period. The medical 
portion of the budget is more complex, requiring line-item detail for salaries, 
supplies, overhead and other variable items. The budget should be checked out 
by someone with experience in the fiscal affairs of similar programs. 

The budget is converted into a capitation. This requires estimating the 
membership level at which it is “reasonable” to expect operating revenues and 
expenses to come into balance-the “break even point”. 

Determining an appropriate breakeven point is a difficult but critical 
operation, because capitations (and ultimately premiums) are generally based 
on the per capita operating cost at a breakeven level of membership, which Dr. 
Feldstein calls the “future cost”: the per unit cost with increased size; or 
“economy of scale”. The break-even membership level can vary, but it is likely 
to be close to the membership level for which the facility was designed so that 
the maximum contribution to fixed costs is made. 

In considering the economies of scale and fixed variable cost, there are 
somewhat different views on the question of optimal plan or facility size. At 
what membership level are there economic advantages because of sheer vol- 
ume? Is there a point of diminishing returns where added membership results 
in economic disadvantage? Not many years ago, 20-30,000 members was the 
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rule-of-thumb size for a free-standing group practice facility. Today, some 
plans feel that 35-50, 000 members are required to support a facility. Generally 
speaking, a facility’s size should be such that capitation levels can provide a 
maximum contribution to the Plan’s “fixed costs”. Radiological equipment, 
for example, is usually the most expensive element of non-personnel fixed cost. 
Whether through the amortization of purchased equipment or the rental cost 
of leased equipment, the annual cost of maintaining this service (usually requir- 
. ing two units to accommodate high-turnover routine films and slow-tumover 
special films) would require the close to 35,000 membership level that such 
equipment could handle at capacity. Similarly, efficient utilization of the com- 
plement of staff, most particularly specialty physician staff, requires a member- 
ship level of about 30-35,000. 

Beyond the 30-35,000 range, it would seem that the marketplace would 
need to have the potential to generate larger numbers with some degree of 
assurance and rapidity. Providing capacity (with significant components of 
fixed costs) for larger number of members thereby extending the break-even 
point, without the assurance of attaining such membership levels within a 
reasonable period of time, could stretch the Plan’s resources and make solvency 
an elusive goal. On the other hand, a too-low break-even membership level, i.e., 
attempting to recover fixed costs more quickly, could price the program out of 
the marketplace. 

Costs vs. Benefits 

The design of benefit packages also requires considerations related to the 
— cost constraints of the marketplace. Benefit levels should be sufficiently com- 
prehensive to provide desired operating flexibility, but must also remain com- 
— petitive. 

Balancing the desirability of comprehensiveness with the market’s cost 
constraints is a continuous feedback or testing process, generally resulting in 
the need to determine benefit priorities and/or elements of “over-the-counter 
charges” whereby various portions of health care costs are borne at the time of 
service, rather than through prepayment. This process might be diagrammed as 
follows: 
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The determination of capital cost requirements is fundamental to the 
development of any new plan. The estimation of and the identification of 
sources for, required capitalization are essential Any plan’s capitalization re- 
quirements will be of two parts: 1) start-up, and 2) facility. Start-up cost is 
separate from the “pre-start” or developmental planning costs which have been 
funded largely by public or foundation funds, or buried in other budgets. 

Start-Up Capital 

By whatever name— start-up, inception or seed money-new plans require 
deficit financing to support operations until an adequate level of enrolled 
membership is attained. Determining the amount of required start-up financing 
is very difficult, but the experience of plans initiated over the past several years 
has demonstrated the necessity of providing for substantial standby amounts. 

Projecting monthly membership levels, forecasting revenues based upon 
these projections, and determining the deficits from monthly expenses, pro- 
vides a cumulative (but hopefully declining) picture of need. Opportunities for 
the introduction of staff and facilities, and/or the presence of fee-for-service 
revenues, are among a variety of circumstances that can serve to moderate the 
level of start-up requirements. 

Facility Capital 

The costs for construction or rennovation of a facility, and for furnishings 
and equipment, become easier to estimate than the costs attributable to start- 
up. As in the case of operating costs, the experience of existing plans provides a 
wealth of data from which to draw. Specific facility costs will clearly depend 
upon local land values and labor costs, site characteristics, whether an existing 
facility is to be rennovated or a new one built, whether major equipment is to 
be purchased or leased, and similar considerations. 

Where insurance carriers have been involved in the development of plans, 
their contribution toward these capital needs have ranged from token to sub- 
stantial. If carriers are to play a meaningful role in the future of prepaid plans 
their making available front-end money for both start-up and facilities would 
represent a major contribution. 

2. MANAGEMENT 

It is often said that prepaid group practice most closely relates the 
financing of health care to its organization and delivery. These elements are the 
essence of the management process. The fact that they lack integration in the 
health care system at large must be a major difficulty in the lack of effective 
management in the system and most of its component parts. 
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Prepaid group practice not only permits effective financial management; 
it virtually requires it. Unlike more conventional health providers who can bury 
poor management by either increasing the volume of services provided to cover 
costs, or merely passing along the higher costs to permissive third party payers, 
prepaid group practices must operate within predetermined levels of per capita 
revenues. This has led to a variety of management and organizational disci- 
plines v/liich only effective cost systems can make fully meaningful. The par- 
ticipants then discussed the cost data required from the management aspects of 
health care delivery. 

Budgeting and Reporting 

Any responsible organization operates on the basis of a budget. A pre- 
paid group practice plan’s survival depends on a budget that is carefully devel- 
oped and conscienciously managed. 

The budgetary process, in its broadest sense, is the essence of a plan’s 
development and management. The budget is, after all, a quantitative expres- 
sion of the major components of the operation. It includes the development of 
prospective line-item quantities; but also the on-going review, evaluation and 
correction of actual performance. Quite clearly the management of a program 
via a budget requires the ability to project costs relative to salaries, facilities, 
equipment, supplies, and all of the overhead and indirect cost items that are 
part of a comprehensive health service. 

To be meaningful, actual costs must be generated on the same basis so 
that comparisons of actual to projected can be produced, variances highlighted, 
and corrective action taken if necessary. 

Allocation of Resources 

In contrast to the overall cost objective of determining cost per enrolled 
member rather than cost per specified service, the appropriate management of 
resources does require the development of more detailed cost data for particu- 
lar categories of cost; for, the specific management challenge of prepaid group 
practice is to maintain the flexibility to make the most efficient use of avail- 
able resources or, to put it another way, to assure that in terms of what services 
are provided, where and by whom, care is delivered in the most appropriate 
fashion. 

Most commonly, such cost considerations will become, together with 
various professional and social factors, a part of the process of determining 
whether the plan should introduce new categories of manpower, whether a 
patient should be discharged from a hospital early to an organized home health 
program, etc. Cost data will facilitate such “make/buy” decisions as whether to 
send out laboratory work rather than perform it in the center. 
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3. EVALUATION 



It is tempting to allocate costs in a variety of ways so as to facilitate 
“analysis” or “evaluation”. In a practical sense, however, there may be less need 
for on-going sophisticated costing in a prepaid group practice than in a conven- 
tional fee-for-service operation. For one thing, since reimbursement is not 
made on the basis of a unit of service provided, establishing with precision the 
“real” cost of providing that service so that it is fully recovered in its “price” is 
of little importance. Secondly, although economists would like us to develop 
precise measures of “output” or “outcome”, in the health care field the out- 
come of a related cost often defies precise measure. 

Internal Analyse 

The kind of resource decisions discussed, and particularly those of the 
“make/buy” variety, call for internal cost analyses. In most cases, however, this 
sort of cost data would be required only on a special study basis, not continu- 
ously. It will be necessary to allocate costs, but also to consider which costs 
will be fixed regardless of allocation (i.e., cost which will be incurred even if a 
service under consideration is not performed directly by the Plan) and which 
costs will be incremental. 

The development of new benefits (e.g., prepaid drugs) and the per capita 
rates required to support them, requires the same analytical piocess. Other 
internal analyses, for which cost data would be required on an ad hoc rather 
than an on-going basis, would be the evaluation of a proposed incentive system, 
the introduction of automated screening equipment, or even such administra- 
tive alternatives as the size of an enrollment staff. 



External Comparisons 

One of the great frustrations in attempting to evaluate the relative effec- 
tiveness of prepaid plans is the lack of data comparability. Virtually all plans 
recognize the need to standardize definitions and develop common cost report- 
ing systems. This is a clear prerequisite for the determination of the causes of 
variations in costs among the different plans, or between prepaid plans and 
other organizational forms. , — 

Finally, the participants concluded that in the future it should be pos- 
sible to make comparisons among the various plans; for example, to evaluate 
the costs and effectiveness of the different university health care plans. How- 
ever, a considerable amount of work remains to be done before that can be 
accomplished, because the participants noted that there is still a lack of com- 
mon understanding about such items as defining what actually represents an 
“encounter”. 
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PART III: 



CONFERENCE OUTCOME AND PROPOSALS FOR ACTION 
DISCUSSION SUMMARY 
Chairman: Sidney S. Lee 



A need to continue the exploration of major issues presented at the 
conference was discussed both formally during earlier sessions and informally 
during the recesses. At the final session, participants reached the general con- 
clusion that an ad hoc working committee should be formed to develop com- 
parable methods for evaluating the processes, outcomes, and costs of health 
services rendered by the university medical care plans. Each of the organiza- 
tions agreed to designate representatives to serve on such a committee. This 
group would meet on a continuing basis to design a basic set of tables, and 
work toward agreement on a core set of data to be included in the basic record 
systems of the plans. 

Proposals for Action: The tasks assigned to this committee are as follows: 

A review will be made of all patient records and encounter forms, ques- 
tionnaires, and other pertinent instruments designed by each plan for collecting 
patient data; 

Each representative will submit a list of the information considered most 
important for comparative purposes. Some items will relate to information 
needed to evaluate how a specific plan achieves its own objectives; other items 
will relate to information needed to comparative purposes with other plans; 

Based on the above information a set of dummy tables will be designated 
so that it will be possible to obtain statistical data on a regular basis foi 
management and operational functions and to provide the flexibility needed to 
select specific data for special study. The dummy tables will be used as a means 
for developing comparable data collection based on common definitions and 
objectives. 

Tne tables will be designed to allow for expansion of the data so that 
comparisons of other types of programs, i.e. OEO, 314E, or other health 
systems can be made. It was suggested that the committee explore the interest 
of these groups in participating in this effort. 
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These follow-up actions, for organizing and funding, should proceed as 
rapidly as possible. The committee should start work immediately before exist- 
ing or planned record systems make the above tasks more difficult. 

The Harvard Center for Community Health and Medical Care agreed to 
organize this effort and to seek funding for its support. 

Actions Accomplished: 

An ad hoc Working Group, composed of representatives from each of the 
plans presented at the conference and from other organizations responsible for 
medical care plans was formed. The National Center for Health Services Re- 
search and Development provided funding which made it possible for the group 
to meet over an extended time. 

The initial task was to agree upon a core set of data which each member 
of the group would include in the basic record system. Such an approach 
makes it possible for each plan to take the following steps: establish com- 
parable methods for data collection; obtain statistical data on a regular basis 
for management and operational functions; and provide a means for selecting 
specific data for special studies. A draft set of dummy tables was designed to 
elicit minimal information necessary for day-to-day operating purposes. The 
tables focus on 1) the characteristics of the population, and 2) the utilization 
of services. In discussions relating to the population tables, the group began to 
face the problems of developing a clear concept of the population to which 
each plan offers its services. The issue of denominators for the utilization tables 
prompted agreement among the group on the requirement that enrollee and 
registrant should be differentiated. In order to obtain systematic reporting 
from different medical groups, two similar sets of basic tables have been de- 
signed; one using the enrollee population as the base for computing utilization 
rates; the second using the registrant population as the base. This separation 
will enable plans with similar populations to compare data presented in the 
basic tables. 

The group is working toward agreement on definitions of specific terms, 
classification issues, and on the recording and tabulating of certain procedures. 
The basic set of tables is in final draft; a draft of an encounter form for use 
with the tables has been completed. The tables and the encounter form are 
based on minimal information with the intent that any medical care plan could 
expand the collection of data, within the categories provided, to meet its own 
needs. A first draft is being prepared of a manual of procedures for the guid- 
ance of other plans and programs that wish to include, in their basic record 
system, the core set of data defined by the ad hoc Working Group in the tables 
and the encounter form. 
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APPENDIX 



1. Backf-round Information on University Medical Care Programs 

• Columbia Medical Plan (Johns Hopkins University) 

• Community Group Health Foundation, Inc. (Howard University) 

• Community Health Care Center Plan, Inc. (Yale University) 

• East Baltimore Medical Program (Johns Hopkins University) 

• Harvard Community Health Plan, Inc. (Harvard University) 

• Medical Care Group (Washington University) 

• UCLA Community Health Care Prepayment Plan (University of 
California) 

2. Issues and Problems Relating to the Organization and Financing of the 
Community Health Care Center Plan, G. K. MacLeod, M.D. 

3. Excerpts from Issues and Problems of Marketing: Experience of the 
Columbia Medical Plan, W. F. Towle 



fatfonnation about any of the University Medical Care Plans 
£"■*** Conference, write to the Director of the Plan. Names and addresses 
appear in the list of Conference participants. 
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Background Information for Conference 



March II, 1970 



1. Name of Medical Care Plan: Columbia Medical Plan 

A) Date Center opened: October 1, 1969 

1 . Number of subscribers: 450 members ( 1 50 families) 

B) Date Center began providing limited services: Not Applicable (NA) 

C) Date Center began providing complete services: October 1 , 1969 
1. Number of subscribers: 450 members (150 families) 

D) If Medical Center has not begun Medical Care Plan, please give 
approximate beginning date: NA 

E) Current date: March 11, 1970 and current number of subscribers: 
2609 (890 families) 

2. Co-sponsoring institutions: Johns Hopkins Hospital and Johns Hopkins 
University (plus participating Insurance Carrier, currently only Connecticut 
General Life Insurance Company). 

3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: Columbia Hospital and Clinic;' Foundation is a 
wholly owned subsidiary of Johns Hopkins University and Johns Hopkins 
Hospital, organized as a non-profit hospital corporation. The Columbia 
Medical Group is a physician partnership with the partners being the Dean 
of the Medical School and two Associate Deans of the School of Medicine. 
The Columbia Medical Plan is the responsibility of these two bodies and 
participating insurance carriers. The Board of Columbia Hospital and Clinics 
Foundation is composed of 9 Trustees appointed by members of the 
corporation who are the Joint Committee of Trustees of Johns Hopkins 
University and Johns Hopkins Hospital; 4 Trustees from the University, 4 
from the Hospital, and 1 from the participating carriers. 
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4. Please specify: 



A) Population^) to be served: 1) Current number 2600. 2) Anticipated 
enrollment 3(1', 000. 3) Characteristics, by age and sex, of current 
enrollees Males 43%; Females 57% 0-5 17%; 6-16 21%; 17-35 26%; 
36-65 26%; 65+ 10% 



5. A) Indicate organization^) with major responsibility for obtaining sub- 

scribers (or enrollees): Connecticut General Life Insurance Company 

B) Describe method(s) of marketing the medical care plan: To groups 
through Connecticut General group sales offices, agents, and brokers 
marketing dual choice plan to employed groups. 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: 1. Groups have natural coverage often with 
other carrier; 2. Multiplicity of employers with only few employees in 
area; 3. Supplemental approach is difficult administratively. 

6. Please state specific objectives of the medical care plan. 

1. To develop in a new city a medical care plan based on prepayment and 
group practice for all citizens of the City and surrounding area who 
elected to join. 

2. To develop the plan that would be replicable as an economic health 
service package in other areas of the country. 

3. To assure that the plan was developed in such a manner that the 
University’s interest in teaching and research would not be compro- 
mised, although this interest would be developed within the limits of 
practicality. 



Sources 
of Funds 



Type of 
Payment 



Restrictions 

in 

Eligibility 



Individuals 

Number 

Eligible 



Individuals 

Number 

Currently 

Enrolled 



Conn. Gen. Prepayment 
Life Ins. Co. per capita 



Geographical 6S00 Colum- 2600 



bia population 
plus 65,000 
in surrounding 
area 
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4. It was necessary to assure that no undue financial risk to the Johns 
Hopkins University and Johns Hopkins Hospital occurred. 

5. That the facilities and services provided in Columbia would be related to 
the University Medical Center in East Baltimore on regional basis. 

6. To enlarge the concept of the Columbia Medical Han to include 
community health services and education, as well as personal health 
services, and to build into the plan the capability of having major input 
into community mental health resources and other community based 
programs such as training of paramedical manpower in affiliation with 
local colleges. 



*--~s7 

7. To assure all potential members a true dual choice in medical care by 
encouraging other non-plan health providers in the area and, where 
practical, developing working relationships with them. 
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Background Information for Conference 



March 1, 1970 



1 . Name of Medical Care Plan: Community Group Health Foundation, Inc. 
Neighborhood Health Center-A Group Practice Han 

A) Date Center opened: December 8, 1969 
1. Number of subscribers: 250 

B) Data Center began providing limited services: December 8, 1969 
1. Number of subscribers: 250 

C) Date Center began providing complete services: Not yet available. 

D) If Medical Center has not begun Medical Care Han, please give 
approximate beginning date: NA 

E) Current date: March 1, 1970 and current number of sub- 
scribers: 1,000 

2. Co-sponsoring institutions: CHANGE-Cardozo Heights Association for 
Neighborhood Growth and Enrichment; Howard University; Group Health 
Association of Washington, D.C. 

3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: (a) Board of Directors: 12 members-Huward 
University’s School of Dentistry and Medicine, Group Health Association of 
Washington, D.C., and CHANGE, Inc. Advisory Health Council: 20 
members-prcfessionals who live in the community, ten enrolled consumers, 
(b) Administrative Unit consists of Project Director, Health Service 
Director, Administrative Services Director, Community Relations Director, 
Training Director, Research and Evaluation Director, all who report to the 
Project Director. 
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4. Please specify: 



A) Populations) to be served: 1) Current number 10,000. 2) Anticipated 
enrollment 20,000. 3) Characteristics, by age and sex, of current 
enrollees. 



Sources 


Type of 


Restrictions 


Individuals 


Individuals 


in 

Eligibility 


Number 

Eligible 


Number 


of Funds 


Payment 


Currently 

Enrolled 


OEO 


$1.8M Grant 


OEO guidelines 


Approx. 

15,000 


379 


Medicare 


80% of fee 


Required 






schedule 
after $50 
deductible 


guidelines 




344 


Medicaid 


$20 per 


Required 


Approx. 


267 




visit fee 


guidelines 


10,000 






for medical 


Must be age 








service 


0-21 years or 
be totally 
disabled 







5. A) Indicate organization^) with major responsibility for obtaining sub- 

scribers (or enrollees): CHANGE, INC. refers clients for same catch- 
ment area; Freedmen’s Hospital refers patients from their clinics eligible 
here. 

B) Describe method(s) of marketing the medical care plan: Local word of 
mouth; brochures and flyers; health educators; referral by persons able 
to predetermine eligibility-Freedmen’s Hospital and Washington Hos- 
pital Center. 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: Advantage-persons are already screened for 
eligibility; disadvantage— dissemination of information inconsistent and 
sometimes inaccurate. 

6. Please state specific objectives of the medical care plan. 

1. To improve the health status of the Upper Cardozo residents by 
providing quality comprehensive health care. 

i " 
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2. To narrow the existing gap in tWhealth status of residents of the 
Cardozo area by making services more accessible geographically and 
economically. 

3. To provide services for the prevention and treatment of disease in such a 
way and at such times as to respond to the consumer’s needs for service. 

4. To practice both therapeutic and preventive medicine focusing on the 
patient s general health rather than on his immediate illness. 

5. To provide for improved social welfare of patients. 

6. To attract and train residents from the community to perform jobs in 

the health service field offering placement in positions offering upward 
mobility. 6 F 

7. To of ain the cooperation of other agencies to insure complete health 
services. 



Background Information for Conference 
March 30, 1970 



1. Name of Medical Care Plan: Community Health Care Center Plan 

A) Date Center opened: Not Applicable (NA) 

B) Date Center began providing limited services: NA 

C) Date Center began providing complete services: NA 

D) If Medical Center has not begun Medical Care Plan, please give 
approximate beginning date : Mid- 1 97 1 

Indicate whether full range of services (X) or limited number of services 
( ) will be available at this time; 

E) Current date: March 20, 1970 and current number of subscribers: NA 

2. Co-sponsoring institutions: Yale University— New Haven Medical Center 
through an Affiliation Agreement 

3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: Organized under “An Act Incorporating 
Community Health Care Center Plan, Inc.,” Special Act No. 335 of the 
Connecticut Senate and House of Representatives in General Assembly. 

4. Please specify: 

A) Population(s) to be served: 1) Current number NA. 2) Anticipated 
enrollment app. 30,000. 3) Characteristics by age and sex, of current 
enrollees NA. 
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Sources 
of Funds 


Proposed 
Type of 
Payment 


Restrictions 

in 

Eligibility 


Individuals 

Number 

Eligible 


Individuals 

Number 

Currently 

Enrolled 


Under 

study 


Prepaid 

capitation 

Temporary 

initial 

supplementary 

grant-in-aid 


Dual choice 

Ceiling on 
welfare par- 
ticipation to 
20% of con- 
current 
enrollment 


60, GOO- 
75, 000 

(Potential as 

of 3/30/70.) 


NA 



5. A) Indicate organization(s) with major responsibility for obtaining sub- 
scribers (or enrollees): Community Health Care Center Plan 

B) Describe method(s) of marketing the medical care plan: Plan represent- 
ative working with Health Education Committee of Greater New Haven 
Central Labor Council; possibly others 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: NA 



6. Please state specific objectives of the medical care plan: 

The major objectives of the Plan are: 

1. Service: To provide comprehensive family health care through a group 
practice mechanism, emphasizing the importance of continuity and 
coordination of services in preservation of well-being, in prevention of ill 
health and in treatment of disease; to promote safeguards of high quality 
while seeking economies in the utilization of skilled manpower, inpatient 
facilities and specialty resources; to meet the fiscal needs through 
contractual prepayments and cost reimbursements; and to achieve 
satisfactions for both the population served and the providers of service. 

2. Training: To experiment on the design of group practice teams for 
effective utilization and training of personnel; to train next-generation 
physicians and nursing practitioners for effective participation in a group 
practice setting; and to train personnel for medical care administration, 
health education and community service. 
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3. Research and evaluation: To perform health services research through 
pre- and post-enrollment studies; to evaluate service and training 
objectives through ongoing studies; and, eventually, to undertake special 
administrative and clinical epidemiological studies. 



, 1 r- 
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Background Information for Conference 



March 11, 1970 



1. Name of Medical Care Plan: East Baltimore Medical Program (A Joint 
Venture of the East Baltimore Community Corporation and the Johns 
Hopkins Medical Institutions) 

A) Date Center opened: Not Applicable (NA) 

B) Date Center began providing limited services: NA 

C) Date Center began providing complete services: NA 

D) If Medical Center has not begun Medical Care Plan, please give 
approximate beginning date: Mid 1970 

Indicate whether full range of services (X) or limited number of services 
( ) will be available at this time. 

E) Current date: March 11, 1970 and current number of subscribers: NA 

2. Co-sponsoring institutions: Johns Hopkins Medical Institutions and the 
East Baltimore Community Corporation 

3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: The EBCC consists of 25 members, i2 of 
whom are elected by enrollees in the health care program to represent them, 
6 of whom are Model Cities Health Councils (Area A and B), 2 are members 
of the Johns Hopkins Medical Institutions staff, 5 are at-large members. 
This group makes all of the policies and handles the administrative direction 
of the operation. Under the terms of a joint working agreement with the 
Johns Hopkins Hospital which is the recipient of the service dollars an 
agreed upon program of services, budgets, and personnel practices will be 
administered by the East Baltimore Community Corporation. 
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4. Please specify: 



A) Population(s) to be served: 1) Current number NA. 2) Anticipated 
enrollment 5,000. 3) Characteristics by age, and sex, of current 
enrollees Median age 14, 2/3 household headed by women, all residents 
of low income housing. 



Sources 
of Funds 


Type of 
Payment 


Restrictions 

in 

Eligibility 


Individuals 

Number 

Eligible 


Individuals 

Number 

Currently 

Enrolled 


Mass. Blue 


Prepaid 


Geographical 


20,000 


800 


Cross 10 
Insurance Cos. 


Capitation 


and Groups only 


7,500 


300 


Mass. D.P.W. 


Capitation 


Income and 
Residence 


Not yet signed 




Medicare 

U.S.P.H.S. 


Capitation 
Grant-in- Aid 


Age 

Grant-in-Aid 


2,500 





5. A) Indicate organization(s) with major responsibility for obtaining sub- 

scribers (or eru Jlees): East Baltimore Community Corporation 

B) Describe metnod(s) of marketing the medical care plan: Individual 
enrollment by household 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: NA 

6. Please state specific objectives of the medical care plan. 

This medical care plan is intended to provide us experience in rendering 
comprehensive health care for a defined population as a pilot effort toward 
establishment of a program for 25,000 people. In establishing these and 
other medical care plans, the Johns Hopkins Medical Institutions intends 
that they be used for demonstration purposes to providers and consumers 
alike, teaching areas, and programs in which improvements in the 
organization and delivery of health services can be developed. In this 
specific instance, we hope to be able to show that a higher quality of care is 
provided at lower cost through the mechanism of prepayment into a group 
practice. 
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Background Information for Conference 



February 15, 1970 



1. Name of Medical Care Plan: Harvard Community Health Plan 

A) Date Center opened: October 1, 1969 
1. Number of subscribers: 100 

B) Date Center began providing limited services: October 1 , 1969 
1. Number of subscribers: 100 

C) Date Center began providing complete services: October 1, 1969 
1. Number of subscribers: 100 

D) If Medical Center has not begun Medical Care Plan, please give 
approximate beginning date: NA 

E) Current date: February IS, 1970 and current number of subscribers:- 



2. Co-sponsoring ii^titutions: Harvard Medical School, Beth Israel Hospital, 
Boston Hospital for Women, Children’s Hospital, Peter Bent Brigham 
Hospital 

3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: Organized under Chapter 180 of Massachusetts 
Laws (General Statute for Charitable Trusts) Board of Members: 11 
members— President, Fellows and Faculty Members of Harvard University, 
Harvard Medical School, Harvard Law School— guide and coordinate Plan 
activities. Board of Directors: IS members— Faculty: Harvard Medical 
School, Harvard University, Harvard Law School; and Chiefs (and/or 
Directors): Beth Israel Hospital, Massachusetts General Hospital, Peter Bent 
Brigham Hospital; and Faculty: St. , John’s Seminary-Primary poiicy- 
making body. Board of Advisors: National Representatives (7 members^ of 
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H.E.W., Blue Cross Association, and other major institutions and publica- 
tions-Consultants to review and evaluate the Plan’s development. 
Administrative Unit consists of Exec. Director, Assist. Director, Admin, 
and Clerical Staff, personnel who report through Exec. Director to Exec. 
Committee of Board of Directors. Service Unit consists of two Medical 
Directors, one responsible to Board of Beth Israel Hospital, one to Board of 
Peter Bent Brigham Hospital. Health Center Administrator responsible to 
Exec. Director of the Harvard Community Health Plan. 

4. Please specify: 

A) Population(s) to be served: 1) Current number 1,100. 2) Anticipated 
enrollment 30,000*. 3) Characteristics, by age and sex, of current 
enrollees (As of January 30, 1970) Total-754: Males-274, Females— 
480. Ages 0-5, 73; 6-21 , 104; 22-44, 483; 45+, 94. 

*80% enrolled through Blue Cross and Insurance Companies; 20% from 
low-income neighborhoods enrolled through Medicaid, Medicare, and 
grants. 



Sources 
of Funds 


Type of 
Payment 


Restrictions 

in 

Eligibility 


Individuals 

Number 

Eligible 


Individuals 

Number 

Currently 

Enrolled 


Title XIX 


Prepayment 


Income level, 
illness 


3,500 


NA 


Title XVIII 


Prepayment 


Age, etc. 


250 


NA 


PHS, 314e 


Grant 


None 


} 1,250 


NA 


OEO 


Grant 


None 





5. A) Indicate organizations) with major responsibility for obtaining sub- 
scribers (or enrollees): Harvard Community Health Plan 

B) Describe method(s) of marketing the medical care plan: In-house sales 
force talking to employers about dual choice; explaining the Plan to 
employees; generalized literature geared to all segments of population; 
literature developed specifically for one particular group; “Open House” 
tours for potential subscribers; working with public relations staff on 
information for media, etc. 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: We have not had substantial experience that 
would enable us to answer this question at this time. 
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6. Please state specific objectives of the medical care plan. 



The purposes for which tk jrporation is formed are as follows: 



ERIC 



l 

C 

t- 

}■ 



ft 

t 

b 




The primary purpose of the corporation is to join with the Medical 
School of Harvard University and its affiliated teaching hospitals (herein- 
after called the “Medical Institutions”) to advance the development of 
comprehensive health care and to promote medical education by formu- 
lating a program or programs of prepaid comprehensive health services for a 
subscribing population which will: 

(a) Attempt to create effective and economic means of organizing and 
delivering health care; 

(b) Provide the students, faculties, and staffs of the Medical Institutions 
with an opportunity for studying and teaching comprehensive 
health care; and 

(c) Conduct social and clinical research concerning health needs and 
the effectiveness of health care. 
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Background Information for Conference 



March 3, 1970 



1 . Name of Medical Care Plan: The Medical Care Group of Washington 
University (MCG) 

A) Date Center opened: November 20, 1969 

1. Number of subscribers: 300 families to date 

B) Date Center began providing limited services: Complete services pro- 
vided from starting date 

1. Number of subscribers: 100 families 

C) Date Center began providing complete services: November 20, 1969 
1. Number of subscribers: 100 families 

D) If Medical Center has not begun Medical Care Plan, please give 
approximate beginning date: NA 

E) Current date: March 3, 1970 and current number of subscribers: 300 



2. Co-sponsoring institutions: Washington University School of Medicine and 
the Metropolitan Life Insurance Company 

3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: MCG is operated by the staff of the Division 
of Health Care Research with G. T. Perkoff acting as Medical Director, as 
well as Director of the Division. An advisory board for MCG was appointed 
by the Dean of the Medical School, Dr. M. Kenton King, consisting of a 
representative from each clinical department and one each from Barnes and 
St. Louis Children’s Hospitals. There are no consumers on the Board. The 
board advises on policy and research design, and each member acts as the 
coordinator of whatever clinical care is provided MCG patients by his own 
department. 



families 
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4. Please specify: 



A) Population(s) to be served: 1) Current number 300 families. 2) 
Anticipated enrollment 500 families. 3) Characteristics, by age and sex, 
of current enrollees Adult males— 135; adult females-131; depend- 
ents-302 (these data for 1st and 2nd enrollments only, data not 
available for enrollment #3). Only 3 patients from 1905-08. No 
Medicare enrollees have yet received services in MCG. 



Sources 


Type of 


Restrictions 

in 

Eligibility 


Individuals 

Number 

Eligible 


Individuals 

Number 


of Funds 


Payment 


Currently 

Enrolled 


Metropolitan 


Prepayment 


Must be employed 


500 families 


300 families 


Life Ins. 


for ambulatory 


in Co. with some 






Company 


services by 


basic Met. Group 








special insurance, 
plus research 
grant 


Health Ins., must 
have basic hos- 
pital plan, fami- 
lies selected by 
lot at each en- 
rollment in pre- 
arranged #, re- 
mainder applying 
serve as controls. 
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5. A) Indicate organization^) with major responsibility for obtaining sub- 
scribers (or enrollees): Metropolitan Life Insurance Co. 

B) Describe method(s) of marketing the medical care plan: Individual 
contracts in participating companies: (1) St. Louis District Offices, 
Met. Life Ins. Co.; (2) General Motors-UAW, St. Louis; (3) Monsanto 
Chemical Co. In the case of (1), sales representatives met with district 
managers, plan was offered after general meeting. In the case of (2), 
local UAW chapter chairmen had prerogative for offering plan, did so 
diffidently by bulletin board notice. In the case of (3), method was 
same as for (1), working through head, employee benefits section and 
district personnel officers. 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: (1) and (3) work very well. Method (2) only 
partially successful because plan was not sold; benefits were not 
explained well; plan was added as another option to several existing 
ones. 
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Please state specific objectives of the medical care plan. 

1. To develop a system of family oriented group practice within a specialty 
oriented medical center which will use but not duplicate special skills in 
that center. 

2. To compare in study and control patients, who have the same hospital 
insurance, the effect on hospital use, health level, and patient acceptabil- 
ity of an experimental ambulatory care health insurance when payments 
are based on a fee schedule, but are prepaid according to actuarial 
estimates of use. 

3. To provide a pilot program which will give data about use and cost 
which will be clear enough to: 

a. indicate what measure of savings would have to accrue to the group 
from decreased hospital use for such an effort to be on sound fiscal 
grounds if enlarged. 

b. show whether growth outside the center would be feasible. 

c. provide a basis for a plan to care for indigent patients whose medical 
care could be paid for by public or private third parties. 

d. begin a practice which might become a base for teaching comprehen- 
sive health care. 

e. provide the nucleus around which faculty interested in health care 
research could be assembled. 
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Background Information for Conference 
February 25, 1970 



1 . Name of Medical Care Plan: ULCA Community Health Care Prepayment 
Plan 

A) Date Center opened: Not Applicable (NA) 

B) Date Center began providing limited services: NA 

C) Date Center began providing complete services: NA 

D) If Medical Center has not begun Medical Care Plan, please give 
approximate beginning date: November 1 , 1970 

Indicate whether full range of services ( X ) or limited number of 
services! ) will be available at this time. c 

E) Current date: February 25, 1970 and current number of sub- 
scribers: None 



2. Co-sponsoring institutions: UCLA Hospital and Clinics, UCLA School of 
Medicine 



3. Description of Corporate Structure and specify composition of Board 
and/or Administrative Unit: The Regents of the University of California 
operate the UCLA Hospital which will contract to provide services. 



4. Please specify: 

A) Population(s) to be served: 1) Current number NA. 2) Anticipated 
enrollment 36,000. 3) Characteristics, by age and sex, of current 
enrollees (anticipated) 0-16, 12,446; 16-23, 4,800; 23+, 18,736. 50% 
male, middle class, employed family head, group enrollment. 



157 



152 



Sources 
of Funds 


Type of 
Payment 


Restrictions 

in 

Eligibility 


Individuals 

Number 

Eligible 


Individuals 

Number 

Currently 

Enrolled 


100% Blue 


Prepayment on 


Group enrollment 


NA 


NA 


Cross- Blue 


enrollment unit 


through employ- 






Shield 


basis 


ment 







5. A) Indicate organization(s) with major responsibility for obtaining sub- 

scribers (or enrollees): Blue Cross-Blue Shield 

B) Describe method(s) of marketing the medical care plan: Plan will be 
marketed as a new Blue Cross or Blue Shield alternative model to 
traditional fee-for-service. 

C) List the advantages and/or disadvantages in present method(s) for 
obtaining subscribers: Hypothetical advantage in identification as a 
Blue Cross Plan or Blue Shield Plan. 

6. Please state specific objectives of the medical care plan. 

1. To influence favorably (reduce) incidence and severity of health 
impairment (aggregate) in the population (insofar as it is technically 
possible and economically feasible). 

2. To demonstrate a model of financial and contractual relationships 
between consumers and providers of health services, between consumers 
and a health insurance enrollment intermediary, and between that 
intermediary and providers of health care which will provide incentive 
and evaluative stimuli supportive of goal 1 above. 

3. To provide a laboratory within which alternative patterns of health 
service organization can be tested for their outcomes. 

4. To provide a model for the exploration of service commitments 
associated with the teaching of physicians and training of specialists in 
the health sciences. 



ISSUES AND PROBLEMS RELATING TO THE 
ORGANIZATION AND FINANCING 
OF THE COMMUNITY HEALTH CARE CENTER PLAN 



G. K. MacLeod, M.D. 

At the outset, it is important to make clear the distinction between the 
organizational sponsorship of the New Haven Plan and that of the other 
programs being described here today. The Community Health Care Center Plan 
(CHCCP) is not university sponsored; rather it is the plan of a nonprofit 
community corporation authorized under a special act of the 1967 Connecti- 
cut legislature. CHCCP is formally affiliated with the Yale-New Haven Medical 
Center which appoints two members to the plan’s 36-member Board of 
Directors. Both the plan and the Center share the basic objectives of service, 
training and research. It is important to point out that the plan, in contrast to 
the Medical Center, intends to put service ahead of training and research in any 
ranking of these three functions. 

• With regard to service, the central focus of the plan will be to provide 
prepaid comprehensive family health care, through a group practice 
mechanism, emphasizing the importance of continuity, comprehen- 
siveness and coordination of services. The design of the service 
program also includes the decision to seek economies in the utilization 
of skilled manpower, and to control utilization of inpatient facilities 
through appropriate use of ambulatory diagnostic and treatment 
resources. 

• With regard to training, we hope to experiment with new methods for 
the training of next-generation physicians and nursing practitioners for 
effective participation in a group practice setting; and we also hope to 
provide a suitable training ground or residency in medical care 
administration, health education and community service. 

• The research objective is designed to measure and evaluate the 
performance of both the service and training programs. Because this 
conference was convened primarily to discuss research and evaluation, 
this important objective is discussed separately in the paper entitled, 
“Preliminary Evaluation Plans and Methods of the Community Health 
Care Center Plan in New Haven, Connecticut.” 
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In these remarks, I intend not to avoid the controversies and difficulties 
surrounding the organization of a comprehensive health care plan but, using 
the language of the day, to tell it like it is. 



SERVICE ISSUES 

The issues underlying the need for a new approach to delivering services 
in New Haven are not much different from those in the rest of the country 
although, in this community, there have been previous attempts by consumer 
groups to promote organized health care programs. It comes as no surprise that 
these early efforts in New Haven took place after World War II and paralleled 
the era of growing specialization and rising costs of health care. Ef forts of the 
late 1940’s and the SO’s did not even reach the stage of being stillborn. During 
the past five to ten years, a further attempt to develop a prepaid group practice 
plan showed more promise. It now looks like the lr.bor of the past decade, after 
an unusually long gestation, may soon produce a viable delivery system. 

There is in New Haven, as elsewhere, a public awareness of the increasing 
effectiveness of medical care, and a growing concern for its lack of ready 
availability and its increasing costliness. This awareness has been translated into 
an important new social issue, commonly referred to as the right to medical 
care. And this right is supported by large amounts of money which are being 
poured into our medical care non-system from both private and public sources. 
Furthermore, the inefficiency and inadequacy of coverage under existing 
private insurance arrangements are so costly that most people— even with 
insurance-cannot reliably foresee what their medical expenses will be and thus 
are unable to budget against them. 

For the past 20 years, in New Haven, it has become increasingly difficult 
to obtain comprehensive health or medical care as a result of the growing 
shortage of primary physicians without replacement through an organized 
system of care. This shortcoming is manifested by the growing demand for 
emergency room services, convenience clinics and other types of improvised 
medical care at all hours of the day and night. Moreover, as governmental 
research funds were pumped into the University, the staff and facilities of the 
Yale-New Haven Medical Center grew by leaps and bounds while the delivery 
of medical care did not improve and probably deteriorated. 

It is generally expected that a university medical center of such imposing 
stature as Yale should be able to deliver high quality comprehensive family 
health care to at least part of the community and not just provide health care 
primarily to meet its own need for clinical material to service its teaching and 
research programs. It seems fair to say that the limited delivery of 
comprehensive health services which is performed by the Medical Center 
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beyond its own needs is done only in response to unavoidable pressures from 
the New Haven community. Even when a pilot program for family health care 
was devised at Yale for a small group of medically indigent families, patient 
care was fragmented and limited to one socioeconomic group. 



TRAINING ISSUES 

The absence of a training program at Yale which would prepare health 
professionals to provide comprehensive health care relates directly to the 
Medical Center’s lack of commitment to assume responsibility for delivering 
comprehensive family health care to more than just a few families. 

Needless to say, fragmentation of care results from this policy. Such 
fragmentation results when no one primary physician or health care team 
assumes responsibility for continuing health maintenance and for patient care 
during episodes of sickness or disability. And it results when no primary 
physician coordinates all of the services rendered by consultants, subspecialists 
and ancillary health personnel. 

Under the existing system of medical education at most university 
medical centers, it is deemed sufficient to have young physicians learn the 
practice of medicine by concentrating their training mainly on the hospital care 
of the seriously ill “service” or “ward” patients. This situation is highlighted 
within the Yale-New Haven Medical Center where there is not a single full-time 
primary physician among the full-time faculty of close to 500 specialists. Thus, 
there is a serious deficiency in the curriculum with respect to teaching the 
practice of delivering continuous and comprehensive medical care. 

As a result of deficiencies and limitations in training programs, trainees 
have no choice other than to provide fragmented patient care which has often 
led to serious omissions or duplications of important diagnostic, and thera- 
peutic procedures. In addition, long waits in the emergency room and clinics 
and the lack of amenities have contributed to an outcry against the system. 



SERVICE PROBLEMS 

Most of our problems relating to organizing the service component of 
CHCCP could be resolved by money although this has not always been the 
case. I should like to recount some of the probelms we faced during our early 
planning period and which have now been resolved. 

During the early phase of design of this project, It became readily 
apparent that there were legal constraints against anyone other than physicians 
and medical schools arranging for the availability of medical services in 
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Connecticut. To meet the objective of comprehensive family health care, it 
would have been necessary for a professional group to set up several 
corporations to provide the constituent parts of a comprehensive health plan 
such as we are now designing. Because of the likelihood of opposition to this 
activity and the possibility of ensuing litigation if this action was undertaken 
by a community group, we decided to seek broad enabling legislation to permit 
a nonprofit community corporation to set up a program for providing 
comprehensive family health care. 

With support from the labor movement, the State Health Commissioner, 
the State Welfare Commissioner, the Connecticut Hospital Association, the 
Yale Law School senior faculty, many physicians and other professionals, the 
legislation was enacted and signed into law-but only after a hard fought battle 
against some of the more conservative elements in organized medicine, 
dentistry, pharmacy and optometry. It is worth mentioning that we were not 
opposed in this undertaking by the Connecticut Blue Cross and Blue Shield 
Plans or by the insurance industry. 

The next problem was to achieve a formal affiliation between our newly 
chartered corporation and the Yale-New Haven Medical Center. This entailed 
debate, discussion and decision-making in committee meetings, staff meetings 
and councils of the Medical Center, and again was brought to a successful 
conclusion. 

One of the continuing problems with respect to organizing the service 
component is the difficulty which some laymen have in recognizing the very 
great importance of clinical and professional autonomy in patient care and in 
the organization of the professional staff. This lack of awareness may slow the 
trend toward consumer involvement in making arrangements for medical care 
delivery. During the present period of prerecruitment at CHCCP, some 
interested physicians have expressed concern about their degree of control over 
the clinical and professional aspects of the program as well as its productivity 
and quality of care. 



TRAINING PROBLEMS 

The traditional leadership of the medical education system— in many 
medical centers, but not in all— appears to be opposed or reluctant to accept 
responsibility for the training of primary physicians to deliver comprehensive 
family health care. Efforts to establish such training programs have met with a 
lack of support from the departmental chairmen and their faculties who should 
be responsible for providing this very important training experience. Their 
objections follow these general lines: 

• The demand for primary medical care should be met by 
nonphysicians. The highly trained specialist of today is bored by 
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taking care of problems beneath his level of competence and he needs 
to continue to use and develop his technical skills in order to lead a 
full professional life. 

• Another argument says that an affiliated program, such as.CHCCP, is 
not appropriate for training physicians in delivering care. The medical 
center, with its faculty and staff fully committed to training and 
research, is the only suitable environment for physicians-in-training. 

• The debate continues with the fairly honest statement that training 
primary physicians is not the purpose of the university medical 
center. In the center, physicians are trained to work in academic 
medicine, to do research or to practice subspecialty medicine. 

• A final argument is that comprehensive medical care is already 
provided in most university medical centers. Even though it may be 
fragmented and limited to one socioeconomic class, the essential 
elements of comprehensive care can be learned from this experience. 

Hopefully, many forces, both local and national, will support activities to 
dissuade the faculties from their traditional position, and the chairmen will 
follow the leadership of some deans who have been favorably disposed to 
designing training programs in comprehensive family health care. 

A second problem relating to organizing the training program is the 
allocation of staff, space and time for training in a new facility. Since this kind 
of training has not been provided heretofore, we have been obliged to estimate 
how much additional square footage will be needed for residents to train or for 
medical students to assist health personnel. We anticipate, of course, that 
additional personnel will be required to back up trainees providing health care. 
And of great importance is the allocation of time of staff physicians working as 
preceptors for residents and others in training. 



FINANCING 

We cgme full circle to the omnipresent need for the wherewithal to 
finance the program objectives. Though CHCCP was fortunate to be awarded 
federal support for the early years of service and research, our efforts with 
respect to financing the training objective have not been successful to date. 

The cost of training residents would be partially defrayed by their 
services. To this extent, the burden of costs should be assumed as part of 
prepayment. For that portion of. residency training which benefits the general 
community or the nation, the cost should not be assumed by the enrollees in 
the plan but by funds collected for that purpose from private or governmental 
sources. Similarly, the portion of the research program which relates to the 
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day-to-day administration and operation of the program should be charged to 
CHCCP. However, the cost of research activities of a general nature should not 
be passed on to the enrollees. 

A special problem resulting from the use of a university teaching hospital 
by an affiliated group practice plan is the high per diem cost of an expensive 
educational institution. And this will dictate the need for primary physician 
control over impatient utilization. 
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Excerpts from ISSUES AND PROBLEMS OF MARKETING 
Experience of the Columbia Medical Plan 

as presented to Group Health Institute 
Honolulu, Hawaii 
May 26, 1970 

William F. Towle 

Several principles were adopted in the planning process relative to 
marketing. They constituted the decisions on which marketing was to be 
undertaken and are valid with respect to the course of action delineated for the 
program. 

• The Columbia Medical is intended to be available to all residents of 
Columbia City and the surrounding area on a voluntary, dual-choice 
basis regardless of existing employer-provided or other health insur- 
ance. Based on this, an Enrollment/Service Area was defined within 
which total services under the contract could be assured. Enrollment 
outside the Area was to be restricted as much as possible. 

• We concluded after extensive investigation that multiple carrier 
participation in marketing would produce enrollment more rapidly 
than a single carrier participation or the creation of a new carrier 
mechanism. The adoption of this principle was readily agreed to by 
Connecticut General, and by the other carriers with whom we are 
currently discussing participation. As other plans have also concluded, 
we saw a major advantage in capitalizing on the market held by the 
private health insurance industry. 

• Based on recognition that a large majority of potential plan members 
already had employer-provided indemnity insurance, it was necessary 
to develop, in the initial stages of the plan, special group and 
supplemental packages for marketing by participating carriers through 
routes differing from usual employer and union contracts. We were 
cognizant that as employment-centered sales increased by multiple 
carriers, such supplemental packages might diminish. 

• Services from the plan physicians would be made available primarily to 
members, with fee-for-service care restricted to emergency and urgent 
situations. It was thought that such limitation would improve 
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adoption of the plan by employment-centered groups and assure 
priority of care to members. 

Based on these principles, experience in enrollment penetration by other 
prepayment plans, and the fact that the Columbia area has no hospital and 
only limited medical resources, we projected enrollment of several hundred at 
the inception of the plan, 8,000-10,000 by July 1970, 15,000 by July 1971, 
and gradual increase to 70,000-100,000 by 1980. Our actual enrollment 
experience to date, however, falls short of these projections. And it does so for 
a variety of reasons, the main reason being that of the complexity of marketing 
itself. 

To illustrate, permit me to indicate where enrollment currently stands, 
what factors we-and here I reflect opinions both of the carriers and 
ourselves— recognize as contributing positively and negatively to current 
enrollment, and, finally attempt some possible solutions. 

Current Enrollment 

Current enrollment in the Columbia Plan is about 3600 individuals from 
1200 families. Seventy (70%) percent reside in Columbia; 30% outside. 
Thirty-five (35%) percent of the Columbia residents have enrolled; 65% 
continue enrollment in indemnity plans and presumably obtain medical care 
elsewhere. Of the 3600 enrolled, 40% have done so through a special group 
plan approved by the State Insurance Commissioner that established residents 
of Columbia as a group; the remaining 60% (or about 2200 members) are 
enrolled through employer group cases that Connecticut General has insured 
for health and other benefits. Penetration within the present employer groups / 
ranges from under 5% in one group to 90% in another. 

Our expected enrollment to date is being scheduled for three reasons: 

1. General lack of understanding of the plan, perhaps even its presence, 
by the consumer in the Service Area. Episodic illness and use of medical 
resources in such instances is highly ingrained in consumer habits. Others 
depend totally on their employer providing medical insurance without further 
thought. 

2. Saturation by the one participating carrier of employer cases available 
to him at this stage of development. 

3. Failure until just recently to achieve approval for federal employee 
enrollment. The effective date has yet to be established, but it is expected that 
this will be established shortly. As those in the Civil Service Commission with 
whom we are in contact know, our present enrollment projections include a 
substantial number of federal employees and their dependents. According to 
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estimates, 20-25% of the population in our Service/Enrollment Area are federal 
employees and families. 

And, on the plus side, let me also point out that current enrollment is 
about 30% higher than it would be without the development of the 
supplemental package available, at this point, only to Columbia residents. I 
mentioned earlier the approval of a group plan based on residence in the new 
city, which accounts for 40% of enrollment, or about 1300 individuals. The 
bulk of these are enrolled in a supplemental plan by which the head of 
household is able to retain his employer-provided indemnity plan and purchase 
an add-on package at a reduced premium. Such an arrangement makes him a 
full member and the provider files claims under the indemnity plan for services 
it might cover. Reimbursement via this mechanism is credited against 
capitation payments from the carrier. 

In theory, the supplemental plan works well and does provide a going-in 
mechanism whereby members retain employer contributions and are not 
penalized by joining the prepayment plan. However, the mechanism is 
administratively complicated. As employer cases increase, its use and impor- 
tance will diminish except perhaps in cases of high and low options offered by 
a carrier to a single group. 

Complicating Factors 

Let me now turn to some of the factors that further complicate 
marketing of group practice. Again, these factors are taken from our 
experience; however, I believe they are common to other situations as well: 

1. Multiplicity of Employers: A survey of employment uncovered 
approximately 165 different employers of persons residing in the Service/En- 
rollment Area. These employers range from corporate pants such as Westing- 
house and General Electric, Federal, state, and local governments, to many 
businesses with less than 10 employees. Mechanisms for selling and under- 
writing for groups of all sizes must be found. 

2. Multiplicity of Carriers: The large number of insurers offering medical 
insurance, much of it tied to other pieces of the welfare package, likewise 
complicates the picture. A dual-choice package from a single carrier may 
operate as effectively as two competing carriers, one offering indemnity and 
the other prepayment. Such efforts will be ineffective, however, without 
incentive to the insurance agents and brokers promoting this dual package. 

3. Lack of Dual Choice: The lack of the dual choice in non-union and 
some union organizations likewise inhibits prepayment. A major effort 
extending the dual choice concept into the entire employer and union group 
and non-group sectors is essential. Frankly, in my opinion, development of 
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dual choice under Medicare Pact C and Medicaid are more simple tasks than 
convincing management of its merits. 

4. National Insurance Policies: Large corporations, business, and some 
unions are, in many cases, found to have national contracts negotiated from 
headquarters. In such instances, geographically limited group practice plans 
suffer since headquarters would prefer not to be bothered with administra- 
tively complicating local plans. The recently negotiated “group practice plan” 
option in the General Electric-IUE contract is an example of positive action 
toward local option. 

5. Prevalence of Employment-Centered Medical Insurance: Finally, the 
prevalence of medical insurance based on place of employment creates 
difficulty in fostering special groups offered through such mechanism as 
residence area, combination of small groups into groups sufficiently large to 
achieve group advantages, and the like. I am not advocating reducing employer 
contributions, only in finding new mechanisms other than employment-cen- 
tered insurance as a solution to group practice plan marketing. 

Solutions 

And, finally, possible solutions. There are no pat solutions to a very 
complex situation. The variations are many and the details to implement the 
solutions perhaps as complex as the solutions themselves. Here are a few 
suggestions: 

1. Involvement of private health insurance is mandatory if prepayment is 
to be extended. The efforts and successes of present prepayment plans are 
clear; however, inclusion of carriers who currently have access to a vast market 
are required to expose the potential market in quantum numbers. 

2. Agents and brokers offering clients the dual-choice option must be 
educated with respect to the relative advantages and disadvantages of 
indemnity and group practice, market that which is most appropriate and, if 
necessary— and it may be-be provided an incentive for converting a group from 
indemnity to dual-choice. 

3. The group practice plan itself has an obligation in marketing. The 
carrier cannot be expected to offer competing products. To overcome this, 
providers mus’; work in tandem with the insurance agent or broker, with 
management and union, and, most importantly, with the consumer directly. 

4. Corporate national insurance contracts must be liberalized to permit 
local option and approval of dual-choice programs where appropriate. 

5. Special group plans with Medicare and Medicaid are being detailed. 
These must be completed rapidly as a means of alleviating these serious 
national social needs. However, essential to all programs is the need for 
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reciprocal arrangements between group practice plans or indemnity insurance 
to cover out-of-area emergencies, claims processing, and other factors, unless 
the group practice is willing and legally able to assume such a risk. Other 
special group plans for small groups and individuals, based on residence, must 
also be developed. 

Conclusion 

I have attempted in a few minutes to describe a most complex marketing 
situation by presenting the picture as we see it from a particular vantage point. 
New prepayment plans just getting started and on the architect’s table are, or 
will soon be, caught in the complicated web well-known to the established 
plans. Concerted effort by all engaged in these programs is essential to 
successful expansion of prepayment plans. 
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